New York University Teacher Education Program TEAC Inquiry ...€¦ · 4.2.5c Mean CCT GPA by...

New York University Teacher Education Program TEAC Inquiry Brief

CENTER FOR RESEARCH ON TEACHING AND LEARNING

New York University Teacher Education Program

TEAC Inquiry Brief

Authors:

Robert Tobias Director, Center for Research on Teaching & Learning

Member of the Teacher Education Working Group Member of the Teacher Education Council Clinical Professor of Teaching & Learning

Rosa Maria Pietanza

Coordinator of TEAC Internal Audit Member of the Teacher Education Working Group

Coordinator of NYU School Partnerships Master Teacher

Joseph P. McDonald

Chair, Teacher Education Working Group Member of the Teacher Education Council

Professor of Teaching & Learning

Faculty Approval: May 27, 2011 (Evidence available at the NYU internal Blackboard site: STEINHARDT - TEACHER

EDUCATION ACCREDITATION COUNCIL (ORGSITE.STEINHARDT.TEAC)

Submitted to the Teacher Education Accreditation Council

Date: May 31, 2011 (Final Revision: November 15, 2011)

TEAC Inquiry Brief , November 15, 2011 New York University

i

Contents

1.0 PROGRAM OVERVIEW 1.1 Brief History ........................................................................................................................... 1

1.2 Guiding Philosophy and Orientation ..................................................................................... 2 1.3 Program Areas, Levels, Specialties, and Options .................................................................. 4 1.4 Program Demographics .......................................................................................................... 5 2.0 CLAIMS AND RATIONALE FOR THE ASSESSMENTS 2.1 NYU Teacher Education Claims ............................................................................................ 11 2.2 Rationale for Assessment ....................................................................................................... 13 3.0 METHOD OF ASSESSMENT .................................................................................................. 15 3.1 Domain Referenced Student Teacher Observation Scale (DRSTOS-R) Description ............ 16 3.2 New York State Teacher Certification Exams (NYSTCE) Scores Description .................... 18 3.3 Student Teacher End-of-Term Feedback Questionnaire (ETFQ) Description ...................... 20 3.4 Educational Beliefs Multicultural Attitudes Survey (EBMAS) Description ......................... 21 3.5 Grade Point Averages Description ......................................................................................... 23 3.6 Program Exit Survey Description .......................................................................................... 24 3.7 One-Year Follow-Up Survey Description ............................................................................. 25 3.8 Graduate Tracking Study Description .................................................................................... 26 3.9 Graduates’ Value-Added-Modeling (VAM) Effects on Pupils’ Standardized Test Scores Description ............................................................................................................................. 28 4.0 RESULTS 4.1 Summary of Overall Findings ................................................................................................ 29 4.2 Detailed Results for Each Measure ........................................................................................ 32 4.2.1 DRSTOS-R ................................................................................................................... 32 4.2.2 New York State Teacher Certification Exams (NYSTCE) Scores ............................... 37 4.2.3 Student Teacher End of Term Feedback Questionnaire (ETFQ) ................................ 41 4.2.4 Educational Beliefs and Multicultural Attitudes Survey (EBMAS) ........................... 42

4.2.5 Grade Point Averages (GPA) ...................................................................................... 44 4.2.6 Program Exit Survey .................................................................................................... 46 4.2.7 One-Year Follow-Up Surveys ..................................................................................... 49 4.2.8 Graduate Tracking Study .............................................................................................. 51 4.2.9 Graduates’ Value-Added Effects on Pupils Standardized Test Scores ....................... 54 5.0 DISCUSSION AND PLAN

5.1 Discussion .............................................................................................................................. 55 5.2 Plan ......................................................................................................................................... 56

5.2.1 Progress on the First IB Plan ........................................................................................ 56 5.2.2 The Plan Going Forward .............................................................................................. 58

6.0 REFERENCES ........................................................................................................................... 60


ii

7.0 APPENDICES Appendix A: Internal Audit ....................................................................................................... 64 Appendix B: Capacity ................................................................................................................ 108 Appendix C: Qualifications of the Faculty ................................................................................. 125 Appendix D: Program Requirements .......................................................................................... 138 Appendix E: Evidence ................................................................................................................. 144 Appendix F: Local Assessments ................................................................................................ 151 Appendix G: Accreditation of Professional Education Programs ................................................ 174


iii

LIST OF TABLES

Table Page 1.3 Registered Teacher Education Curricula Options 2010-2011 5 1.4a Enrolled Students by Gender, Ethnicity, Registration Status, Full-time, Part-Time Fall 2010 6 1.4b New Student Enrollment in Teacher Education Undergraduate and Graduate 2006-2010 7 1.4c Total Enrolled Undergraduate Majors, Fall 2010 8 1.4d Total Enrolled Graduate Majors, Fall 2010 8 1.4e Steinhardt Full-time and Adjunct Faculty, Fall 2008 – Spring 2011 9 1.4f Teacher Education Faculty by Rank, Gender & Ethnicity, 2010-2011 10 2.1 NYU Claims Mapped to TEAC and NYS Standards 11 2.2 Summary of Measures, Standards, and Participants for Inquiry by Claim 14 3.3 Alignment between ETFQ items and NYU’s claims 21 4.1 Summary of Assessments of Claims 31 4.2.1a Percentage of Late-Placement Student Teachers Meeting Standards on the

Domain Referenced Student Teacher Observation Scale Revised (DRSTOS-R) by Academic Year 34

4.2.1b Summary of Performance on DRSTOS-R Total Scores for Student Teachers in Their Last Placements by Program Certification Areas, Fall 2006 – Spring 2010 37 4.2.2 Mean Scaled Scores, Effect Sizes, and Passing Rates for Steinhardt Teacher

Education Graduates on New York State Teacher Certification Exams (NYSTCE): Graduates from 2006 – 2010 39

4.2.3 Mean Scores on the Claim Scales on the End of Term Feedback Questionnaire for

Steinhardt Students in Their Final Student Teaching Placement: Classes of 2006 and 2010 42

4.2.4 Mean Scores and Performance against Program Standards on the EBMAS for

BS and MA Steinhardt Teacher Education Program Completers in the Class of 2010 44


iv

4.2.5a Mean GPAs of NYU BS Teacher Education Graduates by Claims Degree,

Classes of 2006 – 2010 45

4.2.5b Mean GPAs of NYU MA Teacher Education Graduates by Claims Degree, Classes of 2006 – 2010 45

4.2.5c Mean CCT GPA by Certification Area Program, Classes of 2006 – 2010 46 4.2.6 Numbers and Percents of Steinhardt Teacher-Education Program Completers Who

Reported on the Program Exit Survey that Their Programs Prepared Them Very or Moderately Well to Begin Teaching, Classes of 2009 and 2010 48

4.2.7 Numbers and Percents of Steinhardt Teacher-Education Program Completers Who

Reported on the One-Year Follow-Up Survey that Their Programs Had Prepared Them Very or Moderately Well to Begin Teaching, Classes of 2007 – 2009 50

4.2.8a Comparison of the Demographics of NYC Schools in which NYU Graduates First

Taught and All NYC Schools Disaggregated by School Type, Sept.2004 – Sept. 2008 Graduates 52

4.2.8b Retention Status and Years of Teaching for Steinhardt Graduates Who Began

Teaching in New York City Public Schools within One Year of Graduation, Classes of 2004 – 2008 (including Sept. 2008 graduates) 53

4.2.9 Mean Actual-Versus-Expected ELA and Math Test Gains for Pupils of Teacher

Education Graduates and Their Percentile Rank among All District Teachers with Similar Years of Experience (NYU Graduates Teaching ELA in Grades 4 – 8 During 2008) 55

A.1 TEWG-Generated Answers to Internal Auditors’ Questions 94 B.1 [Table 5 data]: Capacity for Quality: A Comparison of Program and Institutional

Statistics 110 B.2 [Table 6 data]: Capacity for Quality: Intra-Institutional Statistics 117 B.3 Steinhardt Full-Time Teacher Education Faculty by Rank,

Fall 2010 – Spring 2011 122 B.4 Comparison of Full-Time Steinhardt and University Faculty by Rank

2009 – 2011 123 B.5 Comparison of Full-Time Steinhardt and University Faculty by Gender and

Ethnicity 2008 – 2011 124


v

B.6 Comparison of Faculty Salary (Mean) for Steinhardt and the University 2008-2010 124

C.1 Full Time Teacher Education Faculty 2010 – 2011 125 D.1.1 Graduate and Undergraduate NYS Registered Teacher Education Curricula

Options with Credit and Course Requirements 141 G.1 Accreditation of NYU Professional Education Programs 174


vi

LIST OF FIGURES 4.2.1a By-Item Percentage Passing for BS Late-Placement Student Teachers,

Fall 2006 – Spring 2010 35 4.2.1b By-Item Percentage Passing for MA Late-Placement Student Teachers,

Fall 2006 – Spring 2010 36 4.2.2a Mean Scaled Score for Most-Frequently Taken NYSTCE Content Specialty

Tests-B.S. Graduates 2006 -2010 40 4.2.2b Mean Scaled Score for Most-Frequently Taken NYSTCE Content Specialty

Tests-M.A. Graduates 2006 -2010 41 A.1 Cycle of Internal Audit 66 A.2 TEAC Internal Audit Chart 71


1

1.0 PROGRAM OVERVIEW 1.1 Brief History

New York University (NYU) is one of the world’s leading research universities. Founded in 1831, it has grown from a student body of 158 in its first semester to more than 50,000 students today, making it also one of the world’s largest universities. It is accredited by the Middle States Association of Colleges and Schools, and is one of only 60 members of the distinguished Association of American Universities. The faculty has over 3,100 full-time members whose research and teaching encompass virtually the entire scope of the arts, sciences, social sciences, and professions. NYU grants more than 25 different degrees, and enrolls students from every state as well as 130 other countries. NYU students attend 18 schools and colleges at six major centers in Manhattan and Brooklyn, as well as a new comprehensive liberal arts college in Abu Dhabi. Manifesting NYU’s ambition to be the first globally networked university, students also study at NYU campuses in London, Madrid, Paris, Berlin, and other European cities, and in Beijing, Shanghai, Accra, Tel Aviv, and Buenos Aires. Despite its vastness, however, NYU is also deliberately de-centralized in its teaching and learning communities, which tend to be small to moderate in size, and centered on faculty-led programs of study which exhibit unique characteristics. In this sense, New York University mimics New York itself – vast city of mostly intimate neighborhoods.

The global ambition of NYU is distinctly 21st century, but it grows from an early seed. NYU founder Albert Gallatin, Secretary of the Treasury in the Jefferson and Madison administrations, aimed to establish "in this immense and fast-growing city,” as he put it, “a system of rational and practical education fitting for all and graciously opened to all." He was inspired in part by the founders of the University of London, with whom he communicated. As they did too, he envisioned a university inspired by and imbued with the intense activity and energy of city life and city commerce. He spoke of NYU as being “in and of the city,” but even then a city linked with other parts of the world.

In 1890, following Gallatin’s logic, at a time of immense ferment in American schooling and in the cultural make-up of New York, NYU opened the first university-based graduate school dedicated to the advanced education of teachers and school administrators. Thus NYU implicitly confronted what was then the prevalent idea – and is lately an emerging one – that universities are not essential to the education of educators, and that practical knowledge alone, rather than integrated practical and theoretical knowledge, is sufficient input for learning to teach. The new school insisted otherwise. It was called the School of Pedagogy, and was the forerunner of today’s Steinhardt School which remains the NYU home of teacher education. Over the course of the twentieth century, the School also became the NYU home of graduate and undergraduate professional education in media, applied psychology, physical and occupational therapy, nutrition, music, and the visual arts. All of these programs emerged from an initial focus on the learning needs of pre-collegiate youth. Meanwhile, the philosophy of teacher education at NYU became gradually steeped in this unique institutional evolution of loosely allied professional programs and academic diversity. It drew as well on the NYU history of linking Gallatin’s “practical and rational” knowledge (or what we would call today practical and theoretical/research-based knowledge), and also on his idea of drawing energy for education from a city well connected with a larger world.


2

1.2 Guiding Philosophy and Orientation of NYU Teacher Education NYU teacher education spans many areas and Steinhardt departments, as discussed

below, and each area has distinguishing values – repeating the NYU organizational pattern mentioned above of breadth mediated by intimacy. These values range from a focus in science education on hands-on “doing” science and informal outside-of-school settings for learning science, to a special emphasis in elementary education on teaching children with disabilities, to a focus in art education on the power of art to advance social justice. At the same time, the NYU Teacher Education Program as a whole has a general pedagogical core (see Appendix D), an organizational core (see below and also Appendix A), and at least five general program core values, as follows:

1. To be in and of the city and engaged deeply in New York schools. 2. To integrate theory and practice pro-actively rather than expect students to do it on their

own. 3. To promote intercultural openness as a tool for teaching – and in this sense to be in and of

the world. 4. To value content knowledge (disciplinary, interdisciplinary, and inter-professional), and

also pedagogical knowledge as crucial for effective teaching. 5. To engage habitually in organizational self-scrutiny, and in the process to contribute to

the knowledge base for effective teaching and teacher education.

The first three of these core values are associated with Gallatin’s founding impulses, while the fourth was evident in the 1890 decision to open a Graduate School of Pedagogy – at a time when situating teacher education within a university context was a bold novelty. The fifth is the product of more recent history – namely, the organizational demands of forming a new Department of Teaching & Learning in the 1980s, the mandatory re-registration of all New York State teacher education programs in the 1990s, and the accountability demands on teacher education that have marked the early 21st century. It is also the product of NYU’s rise as one of the world’s great research universities. These have all pressed the teacher education faculty to articulate the program’s theory of action, to assess the value that the program adds to the education of the students whom NYU teachers teach, and to participate in the effort to build a deep and trustworthy scholarship of teacher education.

All five of these core values have shaped the claims presented in this brief, and account

for numerous features of program design– including recently developed ones. First, the orientation to the city accounts for NYU’s successful project over the last five years to build a large network of partner secondary schools in some of New York City’s poorest neighborhoods, as well as its efforts launched this year to build a comparable network of elementary schools. This orientation also accounts for the emphasis across the program’s core courses and fieldwork on the role that even the poorest communities can play as resources for children, youth, families, and teachers. Finally, it accounts for how NYU discharges its obligations under state registration and TEAC accreditation to educate teachers to be caring professionals.

Second, the orientation toward pro-active integration of theory and practice accounts for the extensive emphasis at NYU on scaffolded fieldwork (including recent changes to pre-student teaching fieldwork), for the substantial number (including recent additions) of field-based seminars and field-based courses, for a faculty whose members across ranks include many with deep roots in and connections to practice, and for an emphasis on helping candidates learn how


3

to reflect in and on action (Schon, 1983, 1987). It also accounts for the program’s recent efforts (in science education, social studies education, and English education) to experiment with what are called residency models, or intense but scaffolded immersions in practice settings (Grossman & Loeb, 2008; NCATE, 2011). Finally, it accounts for NYU’s embrace of what its first TEAC Brief called dynamic tensions at the heart of teacher education – for example, between the demands of content mastery and the uniqueness of each student, and between technical ends and democratic ones (Taub, Tobias & Mayher, 2005). Programs that pro-actively work to integrate theory and practice (as well as campus and field) must learn to live with dynamic tensions.

Third, the orientation to promote intercultural openness as a tool for teaching accounts for

the recent development of cross-national programs at NYU in the teaching of Spanish, Chinese, and French; for an increase in the number of international teacher candidates; for plans to develop study-away opportunities (including fieldwork) in teacher education; and for increased efforts across areas to ensure that NYU teacher candidates gain proficiency in teaching English language learners. It also accounts for efforts across the program core and program areas to ensure that NYU teacher candidates acquire an ethic of cultural respect in the face of probable cultural mismatch between their own backgrounds and the backgrounds of increasing numbers of their students (Rogers-Sirin & Sirin, 2009). Today’s NYU candidates launch their teaching careers amid the greatest mass migration of peoples and contact among cultures the world has ever known (Suarez-Orozco & Suarez-Orozco, 2002). Being “in and of the city” of New York, the city of immigrants, gives NYU a unique position for helping future teachers develop global competence within and beyond US borders.

Fourth, the orientation toward equal attention to content knowledge and pedagogical knowledge - evident in the 1890 launch of teacher education at NYU – has continued to shape it in the century since. For example, NYU’s Steinhardt School houses art, music, and communication programs, as well as teacher education in these areas. It also houses programs in sociology, history, philosophy, psychology and the health sciences that generate and teach knowledge crucial for teaching. This orientation also accounts for NYU’s efforts to build and maintain strong connections between the teacher education faculty and the NYU Arts & Science Faculty. These efforts include the establishment (now seven years old) of a joint oversight committee of members from both faculties (the Teacher Education Council); participation in the Carnegie Corporation’s Teachers for a New Era Network; participation in the Woodrow Wilson Foundation’s scholarship programs in math and history education, and in the Math for America, Noyce Scholarship, and Clinically Rich Integrated Science (CRISP) programs in math and science; new joint efforts with the NYU campuses in Shanghai, Paris, Madrid, and London; and co-teaching efforts between science and science education faculty, and between history faculty and social studies education faculty. Knowing that out-of-school factors can impede or support student learning and development, NYU promotes conversations between Steinhardt teacher educators and Steinhardt health educators, and between Steinhardt faculty overall and the faculty of the Silver School of Social Work. At the same time, NYU experiments with a number of non-university teacher education partners – from The American Museum of Natural History to the Great Oaks Charter Schools – in efforts to create new models of teacher education for the 21st century.

Finally, the orientation toward organizational self-scrutiny accounts for NYU’s expansion and refinement of measures of institutional effectiveness, as this Brief amply demonstrates. It accounts too for the faculty’s enhancement of internal audit procedures, and its plans to expand


4

their use (see Appendix A). It accounts for data-informed program revisions underway – for example, greater emphasis on fieldwork and on topics from classroom management to English language learners to working with parents. And it accounts for the openness of the faculty to systematic research studies of NYU teacher education. These include rigorous internal ones - for example, those by Hummel-Rossi, Tobias & Ashdown (2009), Sirin & Collins (2009), Polleck & Jeffery (2010), and Tobias, et al. (2008, 2009, 2010). And they include large external ones - for example, those by Wyckoff, et al. (2008, 2009), Meier & Crowe (2009), and Poliakoff, Dailey & White (2009). Today, as non-university teacher education programs proliferate, NYU remains an outspoken advocate of the important role that research universities can play in the education of teachers (Brabeck, 2008; Brabeck & Shirley 2003, Alter & Pradl, 2011). The fact that NYU teacher education lives within a major research university gives it another tension to manage, but also a huge opportunity to develop knowledge in teacher education. NYU faculty are now studying how children learn science, how games can be used to teach science, how students transition to high school, how teacher management and teacher care co-mingle in classrooms, how schools manage discipline, how young black men progress through elementary and middle school years, how school-university partnerships function, how the learning of English language learners may best be assessed, how mathematics disability intersects or not with reading disability, and much more of great usefulness to teacher education (see Steinhardt presentations at AERA, 2011).

1.3 Program Areas, Levels, Specialties, and Options

The NYU Teacher Education Program offers curricula leading to New York State initial and professional teacher certification at the baccalaureate degree level and the master’s degree level. These curricula are housed in program areas within three departments: the Department of Teaching & Learning, the Department of Music and Performing Arts Professions, and the Department of Art and Arts Professions. The program overall is housed within the NYU Steinhardt School of Culture, Education, and Human Development, where it is overseen by a University-wide Teacher Education Council, co-chaired by the Deans of Steinhardt and the College of Arts and Science, and by the Teacher Education Working Group (TEWG) – a committee comprised of key faculty and staff who are appointed annually by the Steinhardt Dean in consultation with the Dean of the College of Arts and Science and the chairs of the four departments. There is no Director of Teacher Education at NYU, but the Chair of TEWG plays an overall coordinating role.

Table 1.3 lists the undergraduate and graduate teacher education curricula that are registered with the New York State Education Department (http://www.nysed.gov/heds/IRPSL1.html). These curricula are delivered (depending on program area) in several formats or options. Most have undergraduate and graduate formats, though some have only graduate formats – for example, dance and TESOL. Several offer dual certification – childhood or early childhood teaching with special education, English or social studies with educational theatre, and a foreign language with TESOL.

Some, though not all, of the graduate curricula have a one-year or “fast-track” option, and

some have a recently developed residency option that puts greater emphasis on fieldwork. Most have three-semester “regular track” options, though others typically take four semesters to complete for full-time students. Most enroll part-time students also who complete the program at variable rates. All of these curricular options share a core curriculum as well as the core


5

values described above. All comprise the NYU Teacher Education Program, and are the subject of this Brief.

TABLE 1.3 2010- 2011 Registered Teacher Education Curricula Options

Teaching Educational Theatre, All Grades Teaching Music, All Grades Teaching Dance, All Grades Teaching Art, All Grades Childhood Education Early Childhood Education Teaching English, 7-12 Teaching a Foreign Language 7-12 (Chinese, French, German, Hebrew, Italian, Japanese, Latin, Russian, or Spanish) Teaching Biology 7-12 Teaching Chemistry 7-12 Teaching Physics 7-12 Teaching Earth Science 7-12 Teaching Mathematics 7-12 Teaching Social Studies 7-12 Bilingual Education for Teachers Literacy (B-6, 5-12) Teachers of English to Speakers of Other Languages (TESOL) Special Education Dual Certification: Educational Theatre, All Grades, with English Education 7-12 Educational Theatre, All Grades, with Social Studies Education 7-12 Teaching a Foreign Language 7-12 (Chinese, French, German, Hebrew, Italian, Japanese, Latin, Russian, or Spanish) with TESOL; Childhood Education/Childhood Special Education; Early Childhood Education/Early Childhood Special Education 1.4 Program Demographics Students

The total enrollment at NYU as of spring 2011 is 50,917. The total fall 2011 student enrollment at the Steinhardt School is 6,493. Of this number, 2,562 are undergraduates and 3931 are graduate students. In fall 2010, 1111 Steinhardt students were enrolled in teacher certification programs - 504 at the baccalaureate level, and 607 at the master’s level. The demographic characteristics of these students are detailed in Table 1.4a. The program enrollment by level (2006 to present) is described in Table 1.4b. The total number of undergraduate and graduate students enrolled in individual program options leading to certification is listed in tables 1.4c and 1.4d.


6

TABLE 1. 4a Enrolled Students by Gender, Ethnicity, Registration Status, full Time, Part Time Fall 2010

Program Area Ethnicity Gender Registration Status Total

ASIAN BIRACIAL BLACK HISPANIC NO

DATA WHITE Sum of

TOT FEM Sum of TOT

MALE Sum of FT TOT

Sum of PT TOT

Childhood Education 5 4 3 2 17 28 3 22 9 31 Early Childhood Education 4 2 2 1 1 11 21 0 14 7 21 Literacy (B-6, 5-12) 3 3 15 18 3 14 7 21

Science Education 18 1 2 4 13 22 16 33 5 38

Special Education 69 6 13 31 34 165 300 18 300 18 318 Teachers of English to Speakers of Other Languages

13 1 1 3 1 15 31 3 18 16 34

Teaching a Foreign Language 7-12 32 1 3 7 8 61 95 17 92 20 112 Teaching Art, All Grades 2 5 1 3 10 24 37 8 21 24 45 Teaching Dance, All Grades 2 1 9 1 6 21 38 2 21 19 40 Teaching Educational Theatre, All Grades 4 2 8 3 11 83 85 26 92 19 111 Teaching English, 7-12 13 4 6 9 20 61 93 20 99 14 113 Teaching Mathematics 7-12 22 4 6 9 32 43 30 64 9 73 Teaching Music, All Grades 25 1 3 6 6 55 68 28 86 10 96 Teaching Social Studies 7-12 3 2 3 7 8 35 36 22 45 13 58 Grand Total 212 25 61 82 123 608 915 196 921 190 1111 Notes: Teaching a Foreign Language includes FL/TESOL dual majors and the Teachers of French Dual Program with GSAS. Educational Theatre includes majors with dual status: Educational Theatre and English, and Educational Theatre and Social Studies; Special Education includes dual majors in Special Education/Childhood, Special Education/Early Childhood, and Childhood Special Education and Early Childhood Special Education.


7

TABLE 1.4b New Student Enrollment in Teacher Education 2006-2010

Undergraduate and Graduate

Graduate FAST TRACK ONLY Sum 06 Sum 06 Sum 06 Sum 07 Sum 07 Sum 07 Sum 08 Sum 08 Sum 08 Sum 09 Sum 09 Sum 09 Sum 10 Sum 10 Sum 10

Applied Accepted Enrolled Accepted Enrolled Applied Accepted Enrolled Applied Accepted Enrolled Applied Accepted Enrolled Master’s Initial Certification 343 308 155 249 204 102 218 176 77 219 185 84 182 148 60

FALL NUMBERS Fall 06 Fall 06 Fall 06 Fall 07 Fall 07 Fall 07 Fall 08 Fall 08 Fall 08 Fall 09 Fall 09 Fall 09 Fall 10 Fall 10 Fall 10

Applied Accepted Enrolled Applied Accepted Enrolled Applied Accepted Enrolled Applied Accepted Enrolled Applied Accepted Enrolled Master’s Initial Certification 555 460 175 679 555 218 587 483 174 641 508 191 725 560 207 Master’s Professional Certification 96 87 32 134 124 51 116 100 44 124 105 39 116 99 33

Undergraduate

Fall 2006 Fall 2007 Fall 2008 Fall 2009 Fall 2010 Applied Accept Enroll Applied Accept Enroll Applied Accept Enroll Applied Accept Enroll Applied Accept Enroll

Teaching and Learning Freshmen

516 290 107 470 284 115 393 234 79 418 249 97 420 242 85

Teaching and Learning Transfers

127 87 53 123 75 44 83 61 29 105 68 36 88 72 40

Music Ed & Ed Theatre Freshmen

110 51 25 95 35 23 108 52 27 117 51 25 133 57 28

Music Ed & Ed Theatre Transfers

14 10 7 18 10 8 17 8 5 17 9 7 16 10 4

TOTAL 767 438 192 706 404 190 601 355 140 657 377 165 657 381 157


8

TABLE 1.4c

Total Enrolled Undergraduate Majors Fall 2010 (Program Options Leading to Initial Certification)

Majors Fall 2010 Major Degree Total Educational Theatre, All Grades ETHR BS 40 Teaching Music, All Grades MUED BMUS 64 Teaching English 7-12 ENGE BS 66

Teaching a Foreign Language (all languages combined) BS 24

Teaching Biology 7-12 SBIO BS 9 Teaching Chemistry 7-12 SCHM BS 6 Teaching Physics 7-12 SPHY BS 2 Teaching Earth Science 7-12 SESC BS 1 Childhood Ed/Childhood Special Ed CHSE BS 142 Early Childhood Ed/Early Childhood Special Ed ECSE BS 71 Teaching Mathematics 7-12 MTHE BS 42 Teaching Social Studies: 7-12 SOCT BS 37 TOTAL: 504

TABLE 1.4d

Total Enrolled Graduate Majors Fall 2010 (Program Options & Codes Leading to Initial or Professional Certification)

Majors Fall 2010 Major Degree Total Teaching Art, All Grades ARED BS/MA 3

Teaching Art, All Grades AREI & ARTA MA 35

Teaching Art: All Grades AREP MA 7 Teaching Dance, All Grades DATC MA 32 Teaching Dance, All Grades DATP MA 8 Ed Theatre All Grades & English 7-12 ETED MA 33 Ed Theatre All Grades & Soc Stud 7-12 ETSS MA 8 Educational Theatre, All Grades EDTA MA 30 Teaching Music, All Grades MUSA B.MUS/MA 29 Instrumental Performance/Teaching Music: All grades MSND B.MUS//MA 1 Piano Performance, Teaching Music: All grades, MSPD MA 2 TESOL All Grades TSOG MA 34 Childhood Education CHED MA 31 Early Childhood Education ECED MA 21 Teaching English 7-12 ENGL MA 43 Teachers of English 7-12 ENGP MA 4

Teaching a Foreign Language All languages combined MA 18

Teaching French as a Foreign Language (Joint Degree GSAS) FLTF/FLT MA 20 Teaching a Foreign Language/TESOL FLTS MA 50 Teaching Biology 7-12 SBLY MA 14 Teachers of Biology 7-12 SBLP MA 1 Teaching Chemistry 7-12 SCHY MA 2


9

Teaching Physics 7-12 SPHY MA 3 Childhood Ed/Special Education: Childhood CSEC MA 64 Early Childhood Ed/Special Ed: Early Childhood ESEE MA 32 Special Education: Childhood SECH MA 8 Special Education: Early Childhood SEEC MA 1 Teaching Mathematics 7-12 MTHE MA 27 Mathematics 7-12 MTHP MA 4 Teaching Social Studies 7-12 SSST MA 18 Teachers of Social Studies, 7-12 SSSP MA 3 Literacy B-6 LITB MA 18 Literacy 5-12 LITC MA 3 TOTAL: 607 Faculty

The number of full-time and adjunct (part-time) faculty in the Steinhardt School for the last three years (fall 2008 to spring 2011) is presented in Table 1.4e. Data for the Steinhardt faculty as a whole are presented here because many Steinhardt faculty members teach education students as well as others. The latter especially include faculty affiliated with the departments of Applied Psychology and Humanities and Social Sciences in the Professions.

TABLE 1.4e

Steinhardt Full-time and Adjunct Faculty, Fall 2008 to Spring 2011 Academic Year Full-Time Part-Time

Adjuncts Teaching Assistants/ Graduate Assistants

Total

2008-2009 271 647 184 831 2009-2010 259 695 183 878 2010-2011 262 991* 0 991

*As of 2010-2011, graduate students who teach are appointed as adjunct professors under a program implemented in 2010 called Financial Aid Reform (3 FAR3), hence the sudden elevation of this statistic.

For full-time faculty, teaching load at Steinhardt involves four to six courses per academic year based on tenure/tenure-track or clinical/master teacher status. Part-time (adjunct) faculty teach a maximum of two courses per semester, and complement the full-time faculty, bringing special expertise and experience to the teacher education program, including in some cases current school-based teaching experience. In the 2010-2011 academic year, 48% of teacher education program courses (excluding content area courses and liberal arts courses for undergraduates) were taught by full-time faculty members. See Table D.1.2 on Steinhardt’s password protected website (www.Steinhardt/secure/TEAC).

Although members of the NYU Faculty of Arts & Science are not listed in Appendix B,

Table B.3, it is important to note that they too play a role in NYU teacher education – in both the liberal arts preparation of undergraduate teacher candidates and in their content preparation in certain areas (notably math, science, English, history, and foreign language).

Despite the wide involvement of NYU faculty in teacher education – both within and beyond Steinhardt - it is nonetheless valid to speak of a distinct NYU teacher education faculty. Its members are principally focused on teacher education, and are highly likely in any given semester to teach teacher candidates and in ways deliberately shaped to contribute to the content


10

knowledge and/or pedagogical knowledge these candidates need to become effective teachers. We define this faculty group as all members of the Department of Teaching & Learning; members of the Art and Arts Professions Department who are expressly affiliated with the art education program area; and members of the Music and Performing Arts Professions Department who are expressly affiliated with the program areas of music education, educational theatre, and dance education. This is the group of faculty whose members taught 48% of the program’s pedagogical and pedagogical content courses in 2010-2011, as noted above. It is also the group that met on March 23, 2011, to discuss a draft of this Brief, and that approved the Brief in an electronic vote finalized on May 27, 2011.1

Of the 262 full-time Steinhardt faculty members during 2010-2011, 61 (23%) constitute

the teacher education faculty as defined above. The full-time faculty/student ratio in teacher education is therefore approximately 1:18. Teacher education faculty by rank, gender and ethnicity is presented in table 1.4f.

TABLE 1.4f

Teacher Education Faculty by Rank, Gender & Ethnicity, 2010-2011 Rank Steinhardt Teacher

Education Number (%)

Female Male Asian African American

Hispanic White

Professor 11 (18) 3 8 1 1 9

Associate Professor 13 (21.3) 11 2 1 3 9

Assistant Professor 10 (16.4) 6 4 2 1 7

Clinical Professor 2 (3.3) 2 1 1

Clinical Associate Professor 4 (6.6) 4 4

Clinical Assistant Professor 5 (8.2) 3 2 1 4

Teacher or Master Teacher 12 (19.7) 10 2 2 10

Music Associate Professor 0

Music Assistant Professor 0

Visiting Associate Professor 1 (1.6) 1 1

Visiting Assistant Professor 3 (4.9) 3 3

Other(Assistant Professor/Faculty Fellow

0

TOTAL 61 40 21 2 9 2 48

Table B.3 in Appendix B shows the breakdown of full-time faculty by rank for Steinhardt overall and for the teacher education faculty. It is important to note here that the teacher education faculty is an integral part of Steinhardt, contributing to the broader mission of the School, and that many of its members also teach students in other Steinhardt programs.

1 Progress in conducting the Internal Audit and preparing the Inquiry Brief was tracked by a large group of NYU faculty and administrators by means of an online Blackboard site (home.nyu.edu/ORGSITE.STEINHARDT.TEAC). This group could and did read drafts of the Brief itself as well as all appendices, and suggest changes – either by means of a listserv or by contacting the TEAC Coordinators. However, only the 61 Teacher Education faculty members were eligible to vote.


11

More specific information about the NYU teacher education faculty is included in Appendix C. Comparisons of the Steinhardt faculty with the NYU faculty overall in terms of gender, ethnicity, and rank are included in charts presented in Appendix B. 2.0 CLAIMS AND RATIONALE FOR THE ASSESSMENTS 2.1 NYU Teacher Education Claims As described in section one, the NYU Teacher Education Program has expansive goals that derive from its history and the philosophical orientations of its faculty across numerous program areas. In diverse ways that include the faculty’s research agendas, NYU studies efforts to realize these goals, and adjusts both goals and designs by the light of such studies. The following claims are fundamental to these larger goals, and are the basis of NYU’s assertion of overall program effectiveness, as well as worthiness of accreditation under TEAC Quality Principle One.

Claim 1: NYU Teacher Education Program graduates are competent and qualified in their content knowledge. Claim 2: NYU Teacher Education Program graduates are competent and qualified in their pedagogical knowledge and in their pedagogical content knowledge. Claim 3: NYU Teacher Education Program graduates are competent and qualified in their clinical knowledge, meaning their knowledge of school and classroom contexts and of students. Claim 4: NYU Teacher Education Program graduates are caring professionals. That is, they interact with students in ways that unconditionally accept students as they are, and work confidently and competently to address the students’ educational needs. Table 2.1 maps these claims (as well as the TEAC cross-cutting themes) against the TEAC Quality Principles, and the New York State Teaching Standards.2

TABLE 2.1 NYU Claims Mapped to TEAC and NYS Standards

NYU Claims TEAC Standards State Standards Claim 1: NYU Teachers are competent and qualified in their content knowledge.

Quality Principle I (1.1) Program candidates must understand the subject matter they will teach. Quality Principle I (1.4) Cross-cutting theme 1: Candidates must demonstrate that they have learned how to learn. . . that they have acquired the dispositions and skills. . . that will support life-long learning in their field.

Standard II: Knowledge of Content. . . . Element II.1 (Demonstrate knowledge of the content, including relationships among central concepts, tools of inquiry, and structures and current developments within discipline(s).) Element II.2 (Understand how to connect concepts across disciplines. . .)

2 Adopted by the Board of Regents on January 11, 2011.


12

Claim 2: NYU Teachers are competent and qualified in their pedagogical knowledge and in their pedagogical content knowledge.

Quality Principle I (1.2) Program candidates must be able to convert their knowledge of subject matter knowledge into compelling lessons. . . . Quality Principle I (1.4) Cross-cutting theme 3: Candidates must be able to use appropriate technology in carrying out their professional responsibilities.

Standard II: Knowledge of Content and Instructional Planning. Elements II.2 – II-6 (Relate lessons to real contexts, use broad range of instructional strategies, align to learning standards, provide multiple pathways to achievement, attend to prior understanding, use appropriate materials and resources.) Standard III: Instructional Practice. Elements III.1 – III.6 (Use research-based practices, have high expectations). Standard V: Assessment for Student Learning. Elements V.1 – V.5 (Use multiple measures to assess and document student growth, evaluate instructional effectiveness, and modify instruction through analysis of data).

Claim 3: NYU Teachers are competent in their clinical knowledge, especially of contexts and of students.

Quality Principle I (1.2) Program candidates must be able to convert their knowledge of subject matter into compelling lessons that meet the needs of a wide range of pupils and students. (1.3) Program candidates must be able to teach effectively in a caring way and to act on their knowledge in a professional manner. Quality Principle I (1.4) Cross-cutting theme 2: Candidates must demonstrate that they have learned accurate and sound information on matters of race, gender, individual differences, and ethnic and cultural perspectives.

Standard I: Knowledge of Students and Student Learning, Elements I.1-I.6 (Understand human development; understand language acquisition; know current research on learning; be responsive to diverse needs and interests; appreciate how families, communities, and technologies influence learning.) Standard III: Instructional Practice. Element III.2 (Communicate clearly and accurately with students), element III.6 (Monitor and assess student progress and adapt to student needs.) Standard IV: Learning Environment. Elements IV.1- IV.4 (Create and manage safe, effective, challenging, and supportive learning environments).

Claim 4: NYU Teachers are caring professionals.

Quality Principle I (1.3) Program candidates must be able to teach effectively in a caring way and to act as knowledgeable professionals. Quality Principle I (1.4) Cross-cutting theme 2: Candidates must demonstrate that they have learned accurate and sound information on matters of race, gender, individual differences, and ethnic and cultural perspectives.

Standard VI: Professional Responsibilities and Collaboration. Elements VI.1 – VI5 (Uphold professional standards, collaborate with colleagues and community, communicate with parents, manage non-instructional duties, comply with relevant laws and policies.)


13

2.2 Rationale for Assessment The NYU teacher education faculty has a strong commitment to ongoing evidence-based self-inquiry. In 2004, the faculty established the Center for Research on Teaching and Learning (CRTL) for the purpose of developing a research agenda and building an evidence base for the study of teacher education at NYU. CRTL’s research, disseminated to faculty in internal and external reports and faculty presentations, has informed ongoing faculty program decision-making and has provided most of the data used in this Inquiry Brief. Specific design principles guided the design of the assessment system for this inquiry. First, the inquiry uses multiple measures, methods, and perspectives to strengthen the internal validity of inferences through convergence (see Table 2.2). Second, measures used for determining that students meet institutional standards for graduation and requirements for state certification are included among the multiple assessments. These measures include grade point averages (GPA) and scores on the New York State Teacher Certification Exams (NYSTCE). To increase content validity, the measures are tailored to align with the respective claims. Third, the faculty established rigorous and rational standards for the assessment of the evidence for each claim. For instance, rather than simply using the state passing standard for the NYSTCE exams, NYU set a higher standard—an effect size = 0.80 standard deviations above the passing score, a conventional standard for a large and educationally meaningful difference (Cohen, 1988). Next, the assessment system includes a measure of the actual teaching performance of our students in a clinical setting, the Domain Referenced Student Teacher Observation Scale-Revised (DRSTOS-R), which is based on the work of Charlotte Danielson (1996, 2007). DRSTOS-R is a process and protocol for assessing the pedagogical proficiency of student teachers by collecting evidence from observations of student teachers’ lessons supplemented by pre- and post-lesson interviews, reviews of lesson plans, journals, and reflective essays. Since the items for this scale were selected with the consensus of faculty from a pool developed by Danielson, the scale not only shares the content validity established by Danielson, but also aligns with faculty’s conceptualization of teaching excellence and the claims of the program.

The assessment system also recognizes that the qualities of an effective teacher go

beyond knowledge and skills that can be tested and behaviors that can be observed, Also important are beliefs and attitudes toward teaching, learners and learning, and cultural communities—or what Burant et al. (2007) refer to as teaching dispositions. Although these dispositions are notoriously difficult to measure, the faculty believes they warrant the focus and attention that assessment can provide. Beginning in 2004, the faculty participated in the development of the Educational Beliefs Questionnaire (EBQ) which was designed to measure two dispositional constructs: belief in the efficacy of teaching and social justice/caring. The EBQ was used to assess the developing dispositions of teacher education students from 2004 until 2008. Although the data provided useful feedback, faculty felt that it defined and measured dispositions too narrowly and did not assess a key focus of the program: the development of multicultural attitudes. Therefore, in 2008, CRTL began developmental work on the Educational Beliefs and Multicultural Attitudes Scale (EBMAS), the successor to the EBQ. Evidence from factor analysis and internal consistency reliability analyses have indicated that EBMAS provides valid and reliable measures of teaching efficacy and multicultural


14

TABLE 2.2 Summary of measures, standards, and participants for inquiry by claim

Claims Measures Standards Participants Claim 1: Content Knowledge

DRSTOS-R: Planning & Preparation Mean score of 3.0 for 80% 675 student teachers in Classes of 2007-10

NYSTCE Exam Scores: Content Specialty Tests

Pass rate of 90% and mean score 0.8 SD > passing

1,950 graduates in Classes of ‘06-‘10

Student Teacher End-of-Term Feedback Questionnaire (ETFQ)

Mean score of 4.0 1,309 student teachers in Classes of 2007-10

Content Area GPA Mean GPA of 3.0 All BS & MA graduates 2006-10 Program Exit Survey 80% respond Very or

Moderately Well 209 graduates from May 2009 & 2010

One-Year Follow-Up Survey 80% respond Very or Moderately Well

314 graduates from May 2007 – 2009

Claim 2: Pedagogical and Pedagogical Content Knowledge

DRSTOS-R: Instruction Mean score of 3.0 for 80% 675 student teachers 2007-10 NYSTCE Exam Scores: ATS-W Exam Score Pass rate of 90% and mean

score 0.8 SD > passing 1,937 graduates in Classes of 2006-10

Student Teacher ETFQ Mean score of 4.0 1311 student teachers in 2007-10 Pedagogical Courses GPA Mean GPA of 3.0 All BS & MA graduates 2006-10 Program Exit Survey 80% respond Very or




Claim 3: Clinical Knowledge

DRSTOS-R: Class. Environ & Total Scores Mean score of 3.0 for 80% 675 student teachers in 2007-10 Student Teacher ETFQ Mean score of 4.0 1,311 student teachers in 2007-10 EBMAS: Personal Teaching Efficacy 1 & 2 Mean score of 4.5 175 program completers in 2010 Teaching Skills GPA Mean GPA of 3.0 All BS & MA graduates 2006-10 Program Exit Survey 80% respond Very or




Pupils’ value-added standardized test scores Effect size of 0.20, meaningful but small

190 program graduates teaching ELA and/or math in grades 4 – 8 in NYC (2008)

Claim 4: Caring Professionals and Cross-Cutting Themes

Caring Professionals: DRSTOS-R Classroom Environment Domain

Mean score of 3.0 for 80% 675 student teachers in Classes of 2007-10

Caring Profs.: EBMAS: General Teacher Efficacy

Mean score of 4.5 175 program completers in 2010

Caring Profs.: Exit & Follow-Up Surveys 80% Very or Moderately Well See surveys above Caring Profs.: Commitment to urban schools No stat. sig. differences in

demographics of grads’ schools & all NYC schools

1,024 graduates from 2004-2008 who began teaching in NYC

Caring Profs.: NYCPS teacher retention rates Standards based on City Council investigative report

1,108 graduates from 2004– 2008 who began teaching in NYC

Caring & Multicultural Perspective (MC): EBMAS: MC Attitudes/Social Justice Scale

Mean score of 4.5 175 program completers in 2010

Technology & MC Perspective: Exit & One-Year Follow-Up Surveys

80% Very or Moderately Well See surveys above

Learning how to learn: Cross-cutting themes GPA (BS); Total U/G GPA (MA)

Mean GPA of 3.0 All BS graduates; all incoming MA students: 2006-10

Learning how to learn: NYSTCE Exam Scores: Liberal Arts & Sciences Test

Pass rate of 90% and mean score 0.8 SD > passing

1,850 graduates in Classes of 2006-10

Learning how to learn: DRSTOS-R Professional Responsibilities Domain

Mean score of 3.0 for 80% 675 student teachers in Classes of 2007-10


15

attitudes/social justice. Empirical analysis of data for over 600 students with feedback from faculty led to the setting of the program standard at a mean score of 4.5 on a 6-point scale.

Next, the faculty believes that an assessment system for self-inquiry must include the voices of the teacher candidates themselves. The teacher candidates’ self-perceptions and reports provide information about their confidence, sense of preparation, and satisfaction with their performance and profession. Accordingly, the assessment system surveys students at three key points in their pre-service and in-service training. In order to make the assessment data meaningful for faculty program planning, the evidence has been compared to available norms that provide standards for performance comparisons. Since normative data are not readily available, NYU has had to be creative in finding sources for comparative data. As one example, program exit and follow-up surveys used items from Arthur Levine’s seminal study of schools of education (Levine, 2006), thereby permitting normative comparison to Levine’s study sample. In a second example, the faculty used a report by the New York City Council on teacher retention in the New York City public schools (New York City Council, 2009), to establish standards for a measure of NYU graduates’ successful retention in teaching in inner-city schools. Finally, the assessment system takes advantage of technological advances in electronic data information systems maintained by state and city public education agencies to track graduates reliably into the teaching profession and to obtain measures of their effectiveness in teaching. CRTL has arranged with the New York City Department of Education to match and merge data from their human resources and test data information systems to the electronic records of NYU graduates in order to (1) monitor and assess employment and retention trends in the New York City public schools, and (2) assess the graduates’ teaching effectiveness as measured by the value-added-modeling (VAM) of state-test-score gains of their pupils in English language arts and mathematics. The ultimate goal of the NYU Teacher Education Program is to prepare graduates for entering the teaching profession and successfully educating their students. The tracking data provide direct evidence of the attainment of these longer term goals. Although the use of VAM to evaluate teacher effectiveness is controversial, incorporating VAM in accountability systems as one among several measures of teacher effectiveness is widespread and has received growing support (Steele, et. al., 2010). In this inquiry, VAM is used as Steele recommends, namely as one measure of a graduate’s effectiveness or clinical competence. 3.0 Method of Assessment As described in the previous section on the rationale of the assessment, the self-inquiry used multiple measures and multiple methods to collect evidence. The reliability and validity of all measures are grounded in theory and supported by empirical investigation, as reported in the Results section. This IB for continuing accreditation uses more assessments than the IB for initial accreditation, including the following categories of evidence that were identified in Appendix E of the initial IB as “not available at that time, but planned for future IBs”:

• Category 6. Career retention rate for graduates. Retention rates for graduates teaching in the New York City public schools are reported on pages 53-54.


16

• Category 11. Alumni self-assessment of their accomplishments. Data from the Program Exit Survey and the One-Year Follow-Up Survey indicate graduates’ self-perceptions of the extent to which they were prepared in the essential elements of teaching, pages 46-48 and 49-51, respectively.

• Category 17. In-service teaching. For graduates who were teaching in the New York City public schools (NYCPS), descriptions of graduates’ rates of employment and the types of schools in which they were teaching are reported on pages 51-54.

• Category 18. Standardized test scores and gains of the program graduates own pupils. The value-added state test-score gains of pupils of graduates who were teaching English language arts and/or mathematics in grade 4 – 8 in NYCPS are reported on pages 54-55.

Measures that were planned but not included in this IB are as follows:

• Category 8. Evaluations by employers of graduates. Review of the annual teacher performance reports generated by school principals, including NYU graduates, revealed they were not sufficiently reliable, sensitive, or informative to be used in this self-study. A new teacher effectiveness system that will be phased into New York State public schools beginning in 2011-12 holds more promise for future self-studies.

• Categories 12-16. Graduates’ professional advanced study, leadership, service, and authoring. These data were to be obtained in a five-year follow-up survey of graduates. However, the logistics of implementing this survey with a sufficient response rate for meaningful inferences have proven challenging, and the data were not obtained in time for this IB.

• Category 19. Limited case studies of selected students. These were conducted as part of our research, but are not included here due to IB page limits and the weak value of the data as additional evidence. NYU believes the high cost and low inferential value of more extensive case studies makes them inadvisable.

Details about each method of assessment are described below.

3.1 Domain Referenced Student Teacher Observation Scale (DRSTOS-R) Description The DRSTOS-R (see Appendix F) is an observation protocol for rating the teaching performance of student teachers, based on the work of Charlotte Danielson as presented in her book, Enhancing Professional Practice: A Framework for Teaching (Danielson, 2007). The DRSTOS-R has been used to assess the pedagogical proficiency of NYU’s student teachers with few modifications from fall 2004 through the present. The items of the DRSTOS-R are aligned with national frameworks for teaching, including the widely used standards of the Interstate New Teacher Assessment and Support Consortium (INTASC). The protocol is administered by NYU field supervisors who are required to undergo a full day of training, which includes rating videos of new teachers and moderated discussions aimed at reaching a common understanding of the items and rubrics. Field supervisors must achieve an acceptable level of inter-rater reliability in order to qualify for official administration of the protocol. Items are rated using a four-point scale: (1) Not Yet Proficient, (2) Partially Proficient, (3) Entry-Level Proficient, and (4) Proficient. The 21 items measure four domains of teaching proficiency that are well aligned with the claims. The first is Planning and Preparation (6 items).


17

This domain assesses knowledge of content, state/city standards, curricular resources, assessment, and using data for planning – all of which involve Content Knowledge (Claim1). The second domain is Classroom Environment (7 items). It focuses on the quality of interactions between and among students and teacher, class organization and routines, and behavior management. All involve the skills of a Caring Professional (Claim 4) who has Clinical Competence (Claim 3). The third domain is Instruction (7 items), which assesses the structure and pacing of lessons, the differentiation of instructional goals and methods to accommodate diverse learning styles and needs, and the ability to foster class discussion. These involve Pedagogical and Pedagogical Content Knowledge (Claim 2). Finally, the fourth domain is Professional Responsibilities (3 items). It assesses relationships with school staff and parents, knowledge of the school and community, and the ability to reflect and grow as a practitioner, indicators of Learning to Learn (a Cross-Cutting Theme). The Total Score requires the integration of all skills and domains, an ability that again requires Clinical Competence (Claim 3). A single protocol is used for all program certification areas, consistent with Danielson’s philosophy that the essential skills of teaching are common to all subject areas.3 The target standard is 3, which signifies that the student demonstrates the proficiency expected of beginning teachers. In rating student teachers, field supervisors use all available evidence collected during the entire semester, including formal and informal observations, journals, lesson plans, interviews and discussions, portfolios, etc.

Sample NYU uses the DRSTOS-R for program evaluation, planning, and to inform conversations about the developing proficiency of student teachers. It is not a requirement for graduation but many instructors use it as one source of evidence in the grading of student teachers. The sample included 675 student teachers (235 BS and 440 MA) who were in their final field placements supervised by DRSTOS-trained field supervisors during the 2006-07 through 2009-10 academic years. Empirical evidence of sample representativeness is presented in the results section. Research Design DRSTOS-R protocols are completed summatively by the trained field supervisors of the sample participants at the end of each semester. Summative assessment means that the supervisors are instructed to use the full range of evidence available to them from all of their interactions with the participants during the entire semester, including formal and informal observations, interviews, review of journals, reflections, and portfolios, pupil work, and conversations with cooperating teachers. It should be noted that cooperating teachers are not asked to complete a formal DRSTOS-R because of the absence of opportunities for scorer training. At the end of the semester, field supervisors deliver the completed protocols to CRTL, where the data are key-entered into a database. Diagnostic analysis is conducted on the database to identify outliers for audit and correction.

Rater Reliability To increase rater reliability, all participating field supervisors undergo a full-day training session that includes an assessment of inter-rater agreement. The trainees independently complete protocols assessing videos of lessons by new teachers. A standard set of videos is used with the content of the lessons aligned with the subject area of the trainees. After each video is 3 Note, however, that the DRSTOS-R assessment process requires that the raters have knowledge and expertise in the subject area of the student teachers they are rating and that they apply their expertise in the rating process.


18

rated, the protocols are collected and immediately computer-analyzed to assess the level of inter-rater agreement/discrepancy. The level of agreement is then reported back to the trainees who engage in guided conversation focused on discrepancies in an attempt to reach common understandings of the meaning of the terms in the rubric and the scoring system. Trainees who are consistent outliers are invited to a follow-up session or not certified to administer the protocol. Cronbach’s alpha coefficients applied to a matrix that used raters as items and DRSTOS-R item ratings as cases were in the .73 - .88 range for six of the last eight training sessions, indicating a moderate to high degree of internal consistency among the supervisors’ ratings.

Reliability and Validity of Ratings The theoretical validity of DRSTOS-R ratings as measures of the essential skills of new teachers and their overall pedagogical proficiency is discussed in the rationale for assessment and the description of the protocol above. Internal consistency reliability was assessed through the computation of Cronbach’s alpha coefficients on a sample of data extracted from the operational database and reported in the results section. Empirical validity was examined using the conceptual framework adapted by Onwuegbuzie (2007) that builds on Messick’s theory of validity. Two types of construct validity were explored: structural validity and substantive validity. Structural validity was assessed through the application of exploratory factor analysis to the protocol data for a sample of student teachers from the operational database. The theory underlying the DRSTOS-R is that its 21 items measure a single construct of pedagogical proficiency, organized into four domains for diagnostic/instructional purposes. Accordingly, the hypothesis is that a single factor will explain the majority of variance in the matrix of item inter-correlations, while high internal consistency within domains will support the utility of the domain scores for making inferences about student specific strengths and weaknesses in students’ teaching performance and the related claims. Substantive validity assesses the evidence that the scores are based on the developing pedagogical proficiency of the students. Since this proficiency is fostered by the program courses and field experiences, it is a reasonable hypothesis that the DRSTOS-R scores of students will increase with their time and experience in the program, as evidenced by higher scores for student teachers in their second than first student teaching placements.

3.2 New York State Teacher Certification Exams (NYSTCE) Scores Description In order to receive New York State certification as a teacher, graduates must pass the examinations in their certification areas administered through the NYSTCE program. Elementary education teachers must pass the Liberal Arts and Sciences Test (LAST), the Elementary Assessment of Teaching Skills-Written (ATS-W), and the Elementary Education Content Specialty Test (CST). Secondary education teachers must pass the LAST, the Secondary ATS-W, and the CST for the core subjects they teach. These three sets of exams are described by the New York State Education Department in the New York State Teacher Certification Examination, an Introduction (2001) as follows:

The LAST examines students’ understanding and use of conceptual and problem-solving processes that are characteristic of humanistic, artistic, scientific and mathematical thinking. Both the Elementary and Secondary ATS-W combine complex and challenging selected-response items with a highly targeted extended-response item to yield a deep and


19

broad assessment of teaching knowledge. Both exams evaluate students’ command of instructional planning, assessment and instructional delivery, their understanding of the professional environment, and their knowledge of the learner. The CSTs are substantial examinations that measure acquisition of subject-matter knowledge at a level of understanding required for teaching (p.37).

In this inquiry, the CST was used as a measure of graduates’ content knowledge (Claim 1), the ATS-W as a measure of pedagogical knowledge (Claim 2), and the LAST as a measure of the cross-cutting theme of learning to learn, which requires the ability to apply the liberal arts concepts and skills examined by the LAST.

Sample Students are advised to take the NYSTCE exams regardless of their intention to teach in New York State. Therefore, the faculty reasoned that the sample of examined students would comprise a large and representative sample of NYU’s teacher education graduates. For this study, CRTL included all graduates in the classes of 2006-2010 who had taken one or more of the three categories of NYSTCE tests during or after attending NYU. The size and representativeness of the sample are described in the results section. Research Design Data for the NYSTCE exams were obtained through electronic file matching. Files of all test score data for NYU graduates were downloaded from the test publishers’ website and entered into a database maintained by CRTL. The data are posted several weeks after each test administration, which takes place several times a year. The exam database was matched to a file that contained information on all graduates from the classes of 2006-2010 using social security numbers as the match identifier. CRTL computed mean scaled scores and passing rates for each exam and each class. The standards for attainment of the claims are mean scaled scores at least 0.80 standard deviations higher than the passing score of 220, the equivalent of a large effect size (Cohen, 1988), and a 90% passing rate for each cohort on each exam.

Reliability and Validity Estimates of reliability (Total Test Decision Consistency) for the NYSTCE tests are

typically in the range of 0.95 to 0.9.4 The theoretical validity of the NYSTCE program rests on the New York State Education Department’s claim that the exams are responsive to and consistent with New York State regulations, New York State public school curriculum frameworks and standards, and the curriculum of New York State teacher education programs. New York State regulations, guidelines, textbooks and other instructional materials serve as the foundation for test content and ensure a demonstrable correlation between the regulations, standards, and programs and the NYSTCE. Furthermore, NYSTCE was developed through a collaborative process involving the combined expertise of New York State classroom educators, teacher preparation faculty, psychometric experts, researchers engaged in the study of teaching and learning, and State policy and program personnel. In addition, a Bias Review Committee

4 Downloaded from www.nystce.nesinc.com/PDFs/NYSTCE_Validation_Reliability.pdf on September 24, 2011.


20

was formed to advise the State on issues pertaining to bias and equity, and to review testing materials for potential biases (New York State United Teachers, 2001).

The construct validity of the NYSTCE exams scores was assessed through the comparative analysis of correlations between the exam scores and student GPAs. The analysis tested the hypothesis that the pattern of correlations between and among the various exams and GPAs will be aligned with the content and skills that are assessed by each GPA measure and each exam. These findings were also used to assess the validity of GPA scores. The findings are presented in the results section for the NYSTCE exams.

3.3 Student Teacher End-of-Term Feedback Questionnaire (ETFQ) Description Faculty and staff designed ETFQ (see Appendix F) as an integral component of the evidence base for self inquiry. This online questionnaire elicits feedback from teacher-education students concerning the extent to which they perceive that the student-teaching experience has enhanced their professional knowledge and expertise. The ETFQ format includes a combination of forced-choice and open-ended items divided into three parts. The first part (Items 1 and 2) asks about the school environment, the second part (Items 3 – 14) focuses on the cooperating teacher, and the third part (Items 15 – 25) focuses on the contributions of the student-teacher supervisor. In the context of the student teaching experience, the items ask students to evaluate how well their cooperating teachers and supervisors contributed to their growth as teachers using a five-point, Likert-type scale ranging from (1) “Very Poorly” to (5) “Very Well.” An open-ended prompt asks the students to describe the specific ways in which the cooperating teachers and supervisors helped their professional growth, as well as any specific experiences that were problematic. For this study, data from eight questions on the ETFQ were used to assess three claims (see Table 3.3). Sample All students who are in student-teaching placements are instructed to sign on to the Steinhardt website at the end of every semester to take the online ETFQ. The subset of ETFQ respondents who were in their final student-teaching placements during the academic years 2005-06 thru 2009-10 comprised the sample for this inquiry. The size and representativeness of the sample are discussed in the results section. Research Design At the end of each semester, student teachers are sent email messages directing them to the ETFQ web site. Three follow-up reminders were sent to boost response rates. Response rates and representativeness are reported in the results section. All responses were downloaded into a database maintained by CRTL. Respondents who were in their final student-teaching placement were identified using a question on placement in the ETFQ. Mean responses for the eight items related to the claims were calculated and statistically compared to the program standard of 4.0, a value nominally equivalent to a response of “Well.” Reliability and Validity

While the ETFQ is a self-report measure and subject to the threats to the validity of such assessments, there is an alignment between specific items on the ETFQ and the claims of the inquiry as displayed in Table 3.3. Faculty agreed on the alignment between the claims and the items, thereby confirming the content validity of the ETFQ. The substantive validity of the


21

closed-end responses was supported by the convergence of results with the thematic content of the open-ended responses for samples of participants. Stability of measurement was assessed through the consistency of results across cohorts, as evidence of both reliability and validity. These data are reported in the results section.

TABLE 3.3

Alignment between ETFQ items and NYU’s claims

Claim ETFQ Items

1 Subject-Matter Knowledge

Items 9 & 18

How would you rate your cooperating teacher’s (supervisor’s) assistance in helping you develop content knowledge specific to your field and age group?

2 Pedagogical Knowledge

Items 7 & 15

How would you rate your cooperating teacher’s (supervisor’s) assistance in helping you develop content knowledge specific to your field and age group?

3 Clinical Knowledge

Items 8 & 16

How would you rate your cooperating teacher’s (supervisor’s) assistance in helping you to enhance your teaching practice?

Clinical Knowledge

Items 11 & 19

How would you rate your cooperating teacher’s (supervisor’s) assistance in developing your classroom management skills?

3.4 Educational Beliefs Multicultural Attitudes Survey (EBMAS) Description CRTL developed EBMAS in fall 2009 as a measure of teacher candidates’ developing dispositions toward teaching. EBMAS replaced its precursor, the Educational Beliefs Questionnaire (EBQ), which was administered to Steinhardt teacher-education students from 2004 - 2008. The initial form of EBMAS that was used in this inquiry consisted of 39 items. In addition to the EBQ, EBMAS items were drawn from the Teacher Efficacy Scale (TES) (Gibson and Dembo, 1984) and the Teacher Multicultural Attitude Survey (TMAS) (Ponterotto et al., 1998). Item selection was based on alignment with the goals of the NYU program and the clarity of the items. EBMAS was designed to yield four scale scores: One for general teacher efficacy, defined as the overall belief that teaching can promote the learning of all students regardless of home background or community; a second for personal teacher efficacy, the teacher’s own belief that he or she can educate all children regardless of background; a third for caring/social justice, the belief in the moral and social responsibility of teachers to educate all children equitably; and the last, multicultural attitudes, teachers’ awareness of, comfort with, and sensitivity to issues of cultural pluralism in the classroom. The actual structure and scoring of the survey was determined through factor analysis, as described below and in the results section. The items were statements of beliefs that students responded to using a six-point Likert scale of agreement ranging from (1) Strongly Disagree to (6) Strongly Agree, with the intermediate categories labeled (2) Moderately Agree (3) Slightly Agree and so on. Item statements were counterbalanced, with some stated in the positive form and some in the negative. The sign of the statement was maintained from the source scales.


22

Sample The sample for this inquiry included students who were assessed in three administrations of EBMAS: September 2009, May 2010, and June 2010. The sample was selected to include students who represented the full range of teacher-education students with respect to certification areas, degree, and experience in the program. In total, 671 students were assessed across the three administrations, 327 BS and 338 MA.5 All of these students were included in the analyses of the technical properties of EBMAS, thereby ensuring that the statistical estimates used for the technical work of scale construction and the assessment of reliability and validity were based on large representative samples. The subset of students who were completing their programs in May 2010 was used to assess the claims. The sample is described in the results section. Research Design A paper-and-pencil version of EBMAS was administered to teacher-education students in selected classes by their instructors. The classes included: the New Students’ seminar for BS students and orientation for Fast Track MA students, with the aim of capturing the beliefs of incoming students; student-teacher seminars, which provided an opportunity to assess the beliefs of BS and MA students who were completing the program; and BS students who were mid-way through their programs. This design facilitates the study of the evolving beliefs of students as they progress through the program, as well as the beliefs of program completers, who are the targets for NYU’s claims. The instructors read a script that asked students to be candid in their responses, and ensured them their data would be strictly confidential and reports would include only aggregate data devoid of individual identifiers. Factor analysis was used to identify the items that would be summed to derive scale scores reflecting the underlying conceptual structure of the survey in alignment with the claims. Each scale score represented the mean of the items for the respective scale, with negatively-stated items flipped for uni-directionality. In order to assess the claims, the scale scores were compared to the program standard of 4.5, established by faculty as representing beliefs that are indicative of a positive disposition toward teaching and empirically attainable, based on a preliminary statistical analysis of data. Reliability and Validity The theoretical validity of EBMAS scores as measures of the dispositions to teaching is discussed in the rationale for assessment. Empirical construct validity was examined in two ways. First, structural validity was explored through exploratory factor analysis of data for the full operational sample. Next, substantive validity was assessed through the statistical comparison of the EBMAS scale scores of students in different stages of their program. Since program courses and mentored field experiences are designed to foster the development of dispositions that are favorable to good teaching, it is hypothesized that the EBMAS scores of NYU students will increase as they advance in the program. Internal consistency reliability was assessed through the computation of Cronbach’s alpha coefficients on the data from the operational database. Findings from the empirical analyses of reliability and validity are presented in the results section.

5 Six students did not indicate their degree program.


23

3.5 Grade Point Averages Description Grade point averages (GPA) were among the measures used to assess the attainment of Claims 1-3, as well as the cross-cutting theme of learning how to learn. Across the university, students are graded in each course from A to F with grade-point averages computed on a four-point scale, weighted for course hours. Grades are awarded for achievement of course objectives. The grading criteria are described in the syllabus for each course.

Teacher education students pursuing the BS or B Mus. degrees must major in a subject that is related to their certification area. These courses are taken in the College of Arts and Science and Steinhardt and are designed to build the deep content knowledge, understanding, and skill required for graduates to teach their subjects effectively. The Content Knowledge (CK) GPA for undergraduates is computed as a weighted average of these courses. MA students take their post-graduate courses in Steinhardt and their grades in these courses are used to compute their CK GPA.

Undergraduate students also receive a broad and deep education in the liberal arts and

sciences in large part by meeting the requirements of the Morse Academic Plan (MAP), a common core of courses in the College of Arts and Science. The MAP and the other courses taken at NYU help undergraduates develop a set of intellectual skills, tools and ideas that enable them to learn on their own; knowledge of cultural perspectives, practices and traditions; and facility with the tools of modern technology, the cross-cutting themes. Accordingly, the Cross-Cutting Themes (CCT) GPA is calculated from the aggregate of MAP courses and other contributing courses from both CAS and School of Education. Students pursuing the MA degree took their liberal arts and science courses as undergraduates. This inquiry used their composite undergraduate GPA as a proxy CCT measure.

Students in both BS and MA teacher education programs take courses that comprise a

Pedagogical Core tailored to their certification area(s). These courses were used to calculate Pedagogy Knowledge (PK) GPA (Claim 2) and they include Inquiries into Teaching and Learning, Teaching Students with Disabilities, courses in pedagogical content knowledge, and courses in human development. Grades in student-teaching and practicum courses and seminars were used to compute a Teaching Skills (TS) GPA, a measure of Claim 3, Clinical Practice.

Sample GPAs were computed for all BS and MA members of the classes of 2006-2010. The sample included all students completing a degree program that leads to eligibility for initial New York State teacher certification. This census sample yielded parameters for the full population of graduates. Research Design The Steinhardt Office of Institutional Research provided CRTL with electronic transcript files for all students in the sample. CRTL coded the courses into categories related to the claims as described above. Respective student GPAs were computed for each category by weighting course grades in numbers by course hours. For MA students only, electronic files of undergraduate GPAs were obtained from the Graduate Admissions Office to be used as a CCT GPA. The criterion for each claim was a mean GPA of 3.0 for the respective GPA.


24

Reliability and Validity The content validity of the GPAs rests in the alignment between the courses upon which

they were calculated and the claims they were used to measure. The Steinhardt courses were specifically designed to meet New York State teacher certification requirements. The state requirements (1999) explicitly define the content knowledge, pedagogical knowledge, and teaching skills that must be taught in state-approved teacher education programs. The State Education Department has approved curricula in the Steinhardt School’s BS and MA teacher education programs. The criteria for grades in these courses are clearly stated in the course syllabi and are based on the mastery of course content and learning objectives. A common concern throughout education is grade inflation. A number of researchers have documented the rise in GPA over time and its underlying causes (see Adelman, 1995, 2001; Levine & Cureton, 1998; and Tobias & Miller, 1999). This concern was partially addressed through the comparative validity analysis described above for the validity of the NYSTCE exams. The reliability of the GPAs as measures of the claims was assessed by the consistency of the results across cohorts. The empirical evidence of validity and reliability are reported in the results section. 3.6 Program Exit Survey Description

In May of 2009 and 2010, CRTL conducted online surveys of Steinhardt’s teacher education students who were completing their programs. The purposes of the survey were to evaluate the quality of the teacher education program at Steinhardt, to obtain data that would inform Steinhardt’s efforts at continuous program improvement, and to assess the readiness of our program completers to begin teaching.

The surveys consisted of both Likert-type and open-ended questions organized into the following sections: data about the respondents’ background, including degree, certification, and program area, the respondents’ perceptions of how well their teacher education programs prepared them for teaching, feedback on the strengths and weaknesses of their pre-service programs, and plans for the future. Data from the section measuring perceptions of preparation for teaching were used to assess the program claims. Program completers were asked to use a four-point scale ranging from (4) Very Well Prepared to (1) Not Well At All to report their perceived preparation in 15 areas of essential teaching skill and knowledge. Eleven of these items were drawn from Arthur Levine’s national study of the effectiveness of schools of education (Levine, 2006). The other four referred to skills that faculty identified as key goals of the NYU program but were not included in Levine’s survey. The survey items are closely aligned with the inquiry claims.

Sample The target populations for the surveys consisted of 308 students in May 2009 and 296

students in May 2010 from all certification programs in the Departments of Teaching and Learning, Music and Performing Arts Professions, and Art and Arts Professions. Responses were received from 122 students in 2009 and 110 in 2010, for response rates of 39.6 and 37.2%, respectively. These response rates are within the range described as average or acceptable for


25

online surveys in a paper posted on the University of Texas at Austin website (2007)6. The representativeness of the sample is discussed in the results.

Research Design The survey was administered online using SurveyGizmo web-based survey software. In May 2009 and May 2010, e-mail messages were sent to the graduates directing them to the survey web site. In an attempt to boost response rates, three follow-up reminders were emailed during the survey period. Participation in the study made the respondents eligible for optional entry into a lottery for an iPod nano digital music player. Survey data were downloaded into SPSS data sets for statistical analysis. Reliability and Validity

The reliability of the data was assessed by analysis of the consistency of the results for the two survey administrations. Program faculty reviewed the alignment between the survey items and the claims to ensure the content validity of the results for making inferences about the claims. Empirical validity was assessed through analysis of the convergence between the profiles of responses to the Program Exit Survey and the Levine study sample. Program standards were set using the data from the Levine study as a set of norms. For the Levine sample, the percentages responding that they were “Very Well” or “Moderately Well” prepared by their programs to teach ranged from 27% for Address the needs of students with disabilities to 81% for Understand how students learn. For the 11 items drawn from the Levine survey, the percents of “Very Well” or “Moderately Well” were less than 60% for five items, between 60 and 69% for three items, in the 70% range for two, and over 80% for just one. Using these data as references to set a high, uniform program standard, the faculty set 80% as the program standard for all 15 items. Since the same set of questions was included on the One-Year Follow-Up Survey, the convergence of the results from the two surveys was used to assess the validity of both surveys for making inferences about the claims. Reliability and validity data are reported in the results for the program exit survey. 3.7 One-Year Follow-Up Survey Description CRTL conducted surveys similar to the Program Exit Survey for Steinhardt teacher education graduates in the three years 2007 – 2009. The follow-up survey was intended to assess the perceptions of graduates one year after graduation concerning the extent to which the

6 University of Texas at Austin (2007). Acceptable response rates vary by how the survey is administered: Mail: 50% adequate, 60% good, 70% very good; Phone: 80% good; Email: 40% average, 50% good, 60% very good; Online: 30% average; Classroom paper: > 50% = good; and Face-‐to-‐face: 80-‐85% good. Retrieved July 20, 2009, from http://www.utexas.edu/academic/diia/assessment/iar/teaching/gather/method/survey-‐Response.php


26

program had prepared them to teach. This survey provided information about changes in perceptions between graduation and early experiences in teaching. It also provided another measure for the assessment of the claims. The survey also inquired about the employment of graduates, including their teaching assignments and the locations and types of schools in which they were teaching. The employment data were used to supplement data collected through the graduate tracking study, as described below. Sample For May 2007 graduates, 95 out of 327 graduates responded for a response rate of 29.1%; for 2008, 92 out of 305, 30.2%; and for 2009, 135 out of 308, 43.8%. Across the three classes, 322 graduates completed surveys for a total response rate of 34.3%, in the average or acceptable range for online surveys.7 The representativeness of the sample is discussed in the results section. Research Design The One-Year Follow-Up Survey was administered online to graduates eight months after graduation. Requests were sent to the NYU email addresses of all graduates in January following their May graduation. For each administration, three follow-up emails were sent and lotteries for iPod nanos were used as incentives to boost response rates. Reliability and Validity The methods and evidence for assessing reliability and validity are the same as described above for the Program Exit Survey. The consistency of results across the three cohorts and the analyses of convergent validity are discussed in the results section. 3.8 Graduate Tracking Study Description The data used in this inquiry were obtained by matching files of NYU teacher education graduates to the human resources data system of the NYCDOE. Although this method does not capture data for graduates who enter teaching outside of the New York City public schools, it is able to track accurately and efficiently the large number of students who begin teaching in NYC. Moreover, focusing the tracking study on the NYCDOE is aligned with an important goal of the NYU program—to prepare graduates to staff the NYC schools. Electronic tracking involves sending a data file of NYU graduates to the NYCDOE for matching to their teacher database using social security numbers. Descriptive data are extracted from the matched records, including the dates of initial hire, types of appointment, assignments, schools, and information on retention and attrition. CRTL also obtains demographic data on the schools in which the graduates teach, in order to assess whether graduates are teaching in schools that represent the full range of diversity of NYC, thereby demonstrating their commitment to urban schools, a component of Claim 4, Caring Professional. In order to obtain estimates of teacher employment outside of the NYCDOE, supplementary data were obtained from the One-Year Follow-Up Survey and faculty communications with graduates. The latter is especially important for graduates of the Early

7 Ibid.


27

Childhood and Dual Early Childhood/Special Education programs, many of whom work in private pre-schools that are not included in the NYCDOE human resources system. Sample The sample for the tracking study included 2,819 graduates from the classes of 2004 – 2008; 571 received BS degrees and 2,248 received MAs. The sample included graduates from all teacher education programs that led to eligibility for initial or professional certification in New York State. Although this was a census sample, the data collected was limited to graduates who matched to the NYCDOE human resources system, responded to the One-Year Follow-Up Survey, or responded to email requests from the Directors of the Early Childhood/Dual Early Childhood Programs. Research Design In January 2009, CRTL sent the electronic file for the sample, including names and social security numbers, to NYCDOE for matching and merging with data from their human resources file. NYCDOE returned the merged file to CRTL in March 2009 for analysis. CRTL conducted diagnostic analysis to correct or eliminate outlier data and to ensure that the match was accurate. CRTL then matched the corrected file to NYU files that contained information about the graduates’ teacher education programs. Finally, data from the online Annual School Progress Reports were downloaded from the NYCDOE website (www.schools/nyc.gov) in order to obtain demographics on the schools in which graduates were employed and all NYC public schools for comparative analysis.

Supplementary data on the employment of graduates were obtained from the One-Year Follow-Up Study. The items that were used asked the respondents whether or not they were employed as teachers and, if so, where they were teaching. These data were used to assess the reliability of the electronic data and to obtain estimates of the percentages of graduates who were teaching outside of the NYCDOE. Additional supplementary data were compiled from the emails of graduates to the Early Childhood Program Directors in response to a request for their employment information in spring 2011.

Reliability and Validity The reliability of the tracking data was assessed by inspection of the consistency of

results across classes or cohorts. Employment statistics are analyzed and shared with faculty to inform program planning. However, employment data were not used to assess the claims and, therefore, are not included in this brief. The impact of market forces on employment statistics pose serious threats to the validity of inferences that cannot be controlled or eliminated. The validity of the use of comparative demographics as a measure of Claim 4, Caring Professionals rests on the assumption that commitment to educating all children, especially inner-city children will be manifest in the types of schools in which graduates teach. Working in diverse, inner-city schools is substantive evidence of this commitment. The program standard of no statistically significant differences in the demographics of graduates’ schools of employment and all NYC schools demonstrates a willingness to work in the full diversity of NYC. In addition, retention of graduates who are teaching in NYC schools requires commitment, confidence, knowledge, and skill, the essential characteristics of caring professionals. Faculty used the results of a New York City Council study (July 2009) to establish the thresholds for retention standards. Given the high level of professional caring and competence required to overcome the challenges to sustained teaching in the NYC public schools, exceeding these retention standards is valid evidence in


28

support of Claim 4. Faculty recognizes that market forces and contextual effects are external factors that pose threats to the validity of these measures. However, the faculty believes that the variance in outcomes due to professional caring far outweigh the variance attributable to extraneous factors.

3.9 Graduates’ Value-Added-Modeling (VAM) Effects on Pupils’ Standardized Test Scores Description The VAM score is the average difference between the actual and predicted test scores of a teacher’s pupils. The predicted scores take into account pupil, class, and school variables that affect pupil performance and are beyond the control of the teacher. The assumption is that the difference between actual and predicted performance is attributable to the effectiveness of the teacher. Scores are reported using a proficiency-level scale, which ranges from 1 – 4.5, with a score of 3.0 indicating performance that meets grade-level standards. VAM scores of zero indicate that pupils’ actual performance is exactly as predicted; positive VAM scores indicate actual performance that is above predicted, and is interpreted as a positive teacher effect; VAM scores below zero indicates actual performance below predicted, and is interpreted as negative teacher effect. Sample In 2009, using data from the graduate tracking study (see above), CRTL identified 396 NYU graduates from the classes of 2001 – 2008 who were teaching reading and/or mathematics in grades 4 – 8 in NYC public schools. These graduates were the potential participants for the VAM study. In order for these graduates to be actual participants, they had to be teaching pupils during the 2007 – 08 school year who had taken the state tests in ELA and/or mathematics in 2007 and 2008. Research Design CRTL sent a file of the potential participants to NYCDOE for matching to the VAM test data file and obtained matches for 191 graduates, who became the actual participants in the study. The participants were divided into experience groups based on data from the graduate tracking file. For each experience group, mean actual, predicted, and difference scores (VAM effects) were calculated in ELA and math. The NYCDOE also provided percentile ranks based on the ranking of each graduates’ VAM effect among NYC teachers with similar levels of teaching experience who were teaching the same subject and grade. Median percentile ranks were computed for each NYU experience group, as well as effect sizes based on the ratio of the mean difference score to standard deviation of difference scores. Reliability and Validity

The technical soundness of the VAM score is based on two components: the reliability and validity of the state tests used to calculate VAM and the reliability and validity of the VAM scores for making inferences about teacher effectiveness. The former is reported in detail in the test publisher’s technical reports (CTB, McGraw-Hill, 2009). NYCDOE reports little evidence of the reliability and validity of VAM scores. In an unpublished report shown to NYU researchers, NYCDOE asserts that VAM scores showed moderate correlations with principal ratings of teacher effectiveness. In this inquiry, the consistency of VAM effects over time for the NYU sample was assessed as a measure of the stability of the VAM scores. Substantive construct validity was assessed by the correlation of NYU graduates’ VAM scores with years of


29

teaching experience. It is hypothesized that the correlation will be positive and moderate, thereby indicating that VAM scores systematically behave as one would expect if they were measuring the growing expertise of teachers. The findings are reported in the results section. As a measure of teacher effectiveness, VAM scores are theoretically appropriate for measuring Claim 3. Based on a review of studies using VAM as an outcome measure, faculty set a program standard of a small but meaningful effect size of 0.20 for NYU graduates. 4.0 RESULTS

This section begins with a summary of the overall results of the inquiry organized by claims. The overall findings are followed by a detailed presentation of the results for each measure, including empirical evidence of reliability and validity and the outcomes of the assessment. 4.1. Summary of the Overall Findings

Table 4.1 summarizes the results of the assessments organized around the claims and reported separately for BS and MA students. The table summarizes a total of 78 analyses of the data from 37 different scales associated with nine different measures. Across the 78 analyses, the pre-determined program standards were met or exceeded for 54 (69.2%). Results were mixed for nine (11.5%). And standards were not met for 15 (19.2%). Mixed results are operationally defined as an even split of performance at/above standard and below standard for scales with multiple items. Therefore, overall standards were completely or partially met for nearly four-fifths of the measures/scales. It should be noted that some of the measures are more important to faculty and more reliable and valid than others. With this caveat in mind, the conclusions about the claims that emerge from the weight of the evidence are discussed below.

Claim 1: Content Knowledge: Samples of MA program completers met the program standards in six (85.7%) of the seven analyses, with mixed results on the other. BS students met five standards (71.4%), showed mixed results on one, and did not meet one standard. The latter was the Planning and Preparation Domain of the DRSTOS-R. The DRSTOS-R is an important measure but its validity is threatened by statistical evidence of rater bias and the concerns of some faculty about its appropriateness as a universal measure for all certification areas. BS students received below-standard ratings on the Planning and Preparation items for content knowledge and using assessment for planning. In addition, ratings were low for the certification areas of Math, Dual Early Childhood, Music, and Art. Convergent validity for the difference in performance between BS and MA students is provided by the higher CST exam scores of MA students, generally 3 – 7 scaled scores, and their higher content area GPAs. The survey findings found high levels of confidence in content knowledge preparation for both BS and MA students at graduation, which declined somewhat one year after graduation for BS and MA respondents combined. Taken together, the claim that NYU graduates are competent and qualified in content knowledge is strongly supported by the evidence for MA students and moderately supported for BS students. Claim 2: Pedagogical and Pedagogical Content Knowledge: The results of the analyses for Claim 2 parallel those for Claim 1. Again, the DRSTOS measure, this time the


30

Instruction Domain, showed differential results for BS and MA program completers. The difference in performance was mainly attributable to one item, discussion style, for which MA students performed at standard and BS students performed well below. MA students also showed a meaningfully higher GPA in methods courses than did the BS students. However, overall, the differences in measures were small and limited in scope. Therefore, the evidence strongly supports the validity of the claim of competence in pedagogical knowledge for both BS and MA students. Claim 3: Clinical Knowledge and Skill: The evidence pertaining to Claim 3 is more equivocal than that supporting the other claims, especially for BS students. Of the nine analyses of measures of this claim, BS students met standards on four (44.4%), showed mixed results on two (22.2%), and did not meet standards on three (33.3%). Performance was somewhat stronger for MA students, with standards met for five (55.6%), mixed results for three (33.3%), and performance below standard for one (11.1%). Once again, the performance of BS students on the DRSTOS-R fell below the standard, this time on two scales: the Classroom Environment Domain and the Total Scale. The MA students showed the ability to establish codes of behavioral expectations, maintain an awareness of pupil behavior, and manage the functioning of instructional groups to achieve a level of classroom management. These proved more challenging to BS students. Although both groups met standards for the Teaching Skills (Clinical Practice) GPA, the mean was much higher for MA than BS students, thereby providing convergent validity for both measures. However, the findings from the Program Exit Survey were divergent, with a lower percentage of MA completers feeling prepared to maintain order in the classroom, as compared to BS completers. Both groups met standards on one of the two EBMAS Personal Teacher Efficacy Scales, with MA students expressing confidence that they would be responsible for the success of their pupils, while BS students felt they could solve their pupils’ problems. It should be noted, however, that EBMAS is a new measure and more research is needed to better understand the meaning of its data. Finally, the VAM data were mixed for the combined BS and MA sample, meeting standard for ELA but not math. Taken as a whole, the evidence provides moderate support for the claim of clinical competence. Claim 4: Caring Professional: Both BS and MA graduates performed well on the measures for Claim 4, with both groups meeting standards on five of seven scales (71.4%), although with some differences. One difference was in the Classroom Environment Domain of DRSTOS-R, which is also a measure of Claim 3. Here, the BS students failed to meet standard. The other is the Program Exit Survey, where BS students met standards on all three items compared to only one for the MA students. Data from the Graduate Tracking Study showed that the graduates met standards for serving in high needs NYC public schools and retention in these schools. Overall, despite the small differences, the evidence provides strong support for the claim that both BS and MA NYU graduates are caring professionals. Cross-Cutting Themes: The MA completers met standards for all four of the measures related to life-long learning while the BS completers met standards for three of the four (falling below the standard of 3.0 for Liberal Arts GPA). Both groups met the standards for the three measures of multicultural attitudes, including the item concerning preparation to meet the needs of children from diverse cultures on the Program Exit and One-Year Follow-up surveys, and on the EBMAS Social Justice/Multicultural Attitudes Scale. However, only 61% - 67% of


31

respondents felt prepared to use technology for instruction, falling short of the 80% program standard. Faculty recognizes the need to strengthen the curriculum to address this need.

TABLE 4.1 Summary of assessments of claims

Claims Measure Scale Criteria Results of Assessment BS MA

Claim 1. Content

Knowledge

DRSTOS-R Planning & Prep Mean score of 3.0 for 80% Not Met Met NYSTCE Content Specialty Passing Rate of 90% Met Met NYSTCE Content Specialty Effect Size of 0.80 Met Met ETFQ Content Scale Mean score of 4.0 Met Met GPA Content Courses Mean GPA of 3.0 Met Met Exit Survey Content Knowledge Items 80% Very & Moderately Well Met Met Follow-up Sur. Content Knowledge Items 80% Very & Moderately Well Mixed ** Mixed **

Claim 2. Pedagogical/ Pedagogical

Content Knowledge

DRSTOS-R Instruction Mean score of 3.0 for 80% Not Met Met NYSTCE ATS-W: Pass Rate Passing Rate of 90% Met Met NYSTCE ATS-W: Effect Size Effect Size of 0.80 Met Met ETFQ Pedagogical Scale Mean score of 4.0 Met Met GPA Pedagogical Core Mean GPA of 3.0 Met Met Exit Survey Pedagogical Knowledge 80% Very & Moderately Well Met Met Follow-up Sur. Pedagogical Knowledge 80% Very & Moderately Well Mixed ** Mixed **

Claim 3. Clinical

Competence

DRSTOS-R Classroom Environment * Mean score of 3.0 for 80% Not Met Met DRSTOS-R Total Score * Mean score of 3.0 for 80% Not Met Met ETFQ Teaching Skill Mean score of 4.0 Met Met EBMAS Personal Teacher Efficacy 1 Mean score of 4.5 Met Not Met EBMAS Personal Teacher Efficacy 2 Mean score of 4.5 Not Met Met GPA Teaching Skills Mean GPA of 3.0 Met Met Exit Survey Clinical Skills Items 80% Very & Moderately Well Met Mixed Follow-up Sur. Clinical Skills Items 80% Very & Moderately Well Mixed ** Mixed ** VAM ELA & Math Effect Size of 0.20 Mixed ** Mixed **

Claim 4. Caring

Professional

DRSTOS-R Classroom Environment * Mean score of 3.0 for 80% Not Met Met EBMAS SJ/MA Scale * Mean score of 4.5 Met Met EBMAS General Teacher Efficacy Mean score of 4.5 Met Met Exit Survey Caring Professionals 80% Very & Moderately Well Met Not Met Follow-up Sur. Caring Professionals 80% Very & Moderately Well Not Met Not Met Grad Tracking Types of Schools Employed Students same as all NYC Met Met Grad Tracking Teacher Retention Exceed norms by 5% - 10% Met Met

Cross-Cutting Themes (1) Life-

Long Learning

(2) Multicultural Perspective (3) Use of

Technology

DRSTOS-R Prof. Responsibilities (1) Mean score of 3.0 for 80% Met Met NYSTCE LAST: Pass Rate (1) Passing Rate of 90% Met Met NYSTCE LAST: Effect Size (1) Effect Size of 0.80 Met Met GPA Cross-Cutting Themes (1) Mean GPA of 3.0 Not Met Met EBMAS SJ/MA Scale (2) * Mean score of 4.5 Met Met Exit Survey Diverse Cultures Item (2) 80% Very & Moderately Well Met Met Follow-up Sur. Diverse Cultures Item (2) 80% Very & Moderately Well Met Met Exit Survey Technology Item (3) 80% Very & Moderately Well Not Met Not Met

Follow-up Sur. Technology Item (3) 80% Very & Moderately Well Not Met Not Met * Scale and criterion used for multiple claims ** Mixed is an even split of Met/Not Met criteria for scales with multiple items


32

4.2. Detailed Results for Each Measure 4.2.1. DRSTOS-R Reliability and Validity CRTL conducted an empirical investigation of the reliability and validity of DRSTOS-R on a sample of 447 protocols extracted from the center’s operational database. The protocols were completed by 29 trained field supervisors during the six semesters, fall 2004 through fall 2007. The sample was fully representative of the full population of NYU teacher education graduates, including both BS and MA students from all of the program certification areas. One assessment of the substantive validity of the protocol was conducted on a larger sample. Reliability: The Cronbach’s alpha coefficients for the four domain scores and the total score were as follows: .91 for Planning and Preparation; .91 for Classroom Environment; .83 for both Instruction and Professional Responsibilities; and .96 for the Total Score. These coefficients are evidence of the high internal consistency reliability of the domain and total scores.

Validity: Structural validity was assessed through exploratory factor analysis. A principal components factor analysis revealed that a single factor explained 60% of the variance in the scores of the 20 items for the 432 participants who had complete data.8 The scree plot for this factor analysis indicated strong uni-dimensionality for the scale. This finding is consistent with the hypothesis that a single construct of pedagogical proficiency underlies the items in the four domains of the scale. The potency of this latent construct was indicated by high loadings (0.70 – 0.83) of all items on this single factor. This finding does not negate the utility of the domain scores for instructional and program evaluation purposes. As is evident in the data presented below on the assessment of claims, patterns of differences and similarities in the domain and item scores suggest they are systematically measuring unique and important aspects of teaching and learning beyond the general pedagogical proficiency factor.

An analysis of the substantive validity was conducted on the sample of 1,314 protocols

collected during the period fall 2006 to spring 2010. In this analysis, the scores of students in early and late student teaching placements were statistically compared using t-tests for independent samples. The mean scores of late-placement students were significantly higher statistically than the early-placement students with effects sizes for these differences in the small to moderate range. These differences were observed for all four domains and the total scores and for both undergraduate (BS) and graduate (MA) teacher education students. These results further support the validity of DRSTOS-R as a measure of the developing pedagogical proficiency of teacher education candidates. Assessment of Claims Sample Representativeness: During the four academic years 2006-7 through 2009-10, the DRSTOS-R was administered to more than 1,300 NYU student teachers by 70 trained field supervisors across all of the teacher education program areas in the Department of Teaching and

8 The assessment of the psychometric properties of DRSTOS-R used the 20-item version that was in use until fall 2008 when an additional item was added.


33

Learning, the Department of Art and Arts Professions, and the Department of Music and Performing Arts Professions. Of these student teachers, 675 (235 BS students and 440 MA students) were in their final field placements. The overall sampling fraction was 31.6% of all graduates, 56.1% for the BS graduates and 25.6% for the MA graduates. Program completers from all certification areas were represented in the sample, with the exception of the few listed in the footnote to Table 4.2.1b. The percent of program completers in the sample increased dramatically over the four years of the inquiry. The total participation rate rose from 15.7% in the first two years to 48.9% in the last two. During this period, the BS participation rate increased from 27.6% to 81.2% and MA participation increased from 13.2% to 39.9%. In the last year, two-thirds of all graduates participated in the DRSTOS-R assessment. Accordingly, data for the last two years of the inquiry provide strong estimates of the parameters for the full population of program completers while estimates for the first two years should be interpreted with caution. Overall Results of the Assessment

Table 4.2.1a shows the percentages of student teachers reaching or exceeding beginning teacher proficiency (at least 3.0 on a four-point scale) for the four years of the inquiry, as well as whether the totals across years reach the program standard of 80% scoring at least 3.0. MA students met the standard for all four domains and the Total Score (aligned with claims 1,2,3,4 and the cross-cutting theme of Learning to Learn), with scores that exceeded 80% in Domain 1 (Planning and Preparation) and Domain 4 (Professional Responsibilities), and were statistically equivalent to the standard (within the 95% confidence interval) for Domain 2 (Classroom Environment), Domain 3 (Instruction), and the Total Score. However, the BS students met the program standard only in Domain 4, Professional Responsibilities (Learning to Learn). The pattern of performance over the years suggests that reliability of the data for the first three cohorts of BS students may have been limited by low numbers of participants. The percents at/above level 3 for BS students varied widely during those years, from high to low to high again. The data for 2009-2010, which are based on a large sample (85%) of the BS graduates for the class, is probably a better estimate of the performance of the full graduating class. Nevertheless, performance for BS graduates falls short of the standards regardless of class or sample size. More will be said about this below. Insight into the reasons for the differences in findings between BS and MA students may be gleaned from the study of the scores of late versus early placements cited in the section on validity above. In this study, the mean scores for both BS and MA students were significantly higher statistically in their late versus early placements. However, the MA students scored higher than the BS students on both the early and late placements, suggesting that differences in maturity and experience might be responsible for some of the differences in performance between BS and MA students. Inspection of the percents meeting level 3 for each of the protocol’s 21 items sheds additional light on the relative performance of the BS and MA students (see Figures 4.2.1a and 4.2.1b). Looked at this way, both groups appear to have performed well on most items, but the MA students were clearly superior. The BS students met the 80% criterion on 14 of the essential elements of teaching, scored in the 70% range on four others, and performed in the 60% range on two—Discussion Patterns (Instruction) and Awareness of Pupil Behavior (Classroom Environment). Over 90% met the Level 3 standard for only two items—Teacher/Pupil Communication (Instruction) and Classroom Interaction (Environment). On the other hand, MA students were at or near 90% on 16 skills, in the 80% range on four, and scored below the


34

criterion on only one—Awareness of Pupil Behavior (77.3%). In summary, although the BS students met the criterion in most skills, they did so by a small margin compared to the wide positive margin for MA students.

TABLE 4.2.1a Percentage of Late-Placement Student Teachers Meeting Standards on the Domain Referenced Student Teacher

Observation Scale Revised (DRSTOS-R) by Academic Year Claims Scale Domain Number

of Items Total (N)/

% Meeting Standards

(Mean>=3.0)

2006-2007

2007-2008

2008-2009

2009-2010

Total**

BS Students

1 Planning & Preparation 6

Total (N) 17 37 58 123 235 % Meeting Standards 58.8% 51.4% 69.0% 61.8% 61.7%

3,4 Classroom Environment 7


2 Instruction 5* Total (N) 17 37 58 123 235

% Meeting Standards 70.6% 48.6% 72.4% 62.6% 63.4%

CCT Learning to Learn

Professional Responsibilities 3


3 Total Score 21* Total (N) 17 37 58 123 235


MA Students

1 Planning & Preparation 6


3,4 Classroom Environment 7


2 Instruction 5* Total (N) 21 100 128 191 440


CCT Learning to Learn

Professional Responsibilities 3


3 Total Score 21* Total (N) 21 100 128 191 440


Notes. Scale is (1) Not Yet Proficient (2) Partially Proficient (3) Entry Level Proficient (4) Proficient. The standard for proficiency is 3. *An additional item was added to “Instruction” in fall 2008, increasing the number of items from 4 to 5 ** Values in bold font meet the program standard of 80% >=3; values in bold italics fall within the 95% confidence interval around the standard, which means they are not significantly lower than the standard, p<.025


35

Figure 4.2.1a By-‐Item Percentage Passing for BS Late-‐Placement Student Teachers Fall 2006-‐Spring 2010

81.7

74.8

77.4

84.1

82.7

73.3

80.3

94.8

74.7

80.4

88.0

76.3

61.4

81.1

80.8

80.2

90.1

67.1

87.1

83.5

87.4

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.

Pedagogical Content

Knowledge of Content

Long/Short Term Planning

Constraints on Teaching & Learning

Criteria and Standards

Feedback, Reflection and use for Planning

Teacher Interaction w/ Pupils

Classroom Interaction

Functioning of Learning Groups

Transitions

Materials and Supplies

Mutual Expectations

Awareness of Pupil Behavior

Lesson Structure and Time Management*

Clarity of Goals

Knowledge of Students

Teacher/ Pupil Communications

Discussion Patterns

Relationships w/ Adults

Cultural Context of School & Community

Use in Future Teaching

% >= 3


36

Disaggregation by Program Area

Inspection of DRSTOS-R scores disaggregated by program certification areas sheds some more light on the below-standards performance of BS program completers. As can be seen in Table 4.2.1b, there was wide variability in the performance among the BS programs. In the secondary program areas, the percents at/above level 3 were statistically equivalent to the 80% standard for students in Social Studies and English but far below standard in Math. Performance was also low for the Dual Early Childhood BS program, the MA program in Art, and the BS and MA Music programs, although the low numbers of participants Music raise questions of validity and generalizability. There are several possible factors that might explain these low scores. The Dual Early Childhood faculty has questioned the appropriateness of DRSTOS-R for their area. They have indicated that many items are not aligned with teaching early childhood, and say that they have experienced difficulty collecting evidence on their student teachers’ performance. In this regard, a CRTL study of the correlates of DRSTOS-R scores found statistically significant rater bias (Tobias et. al. 2010). This bias may have contributed to an underestimate of

Figure 4.2.1b By-‐Item Percentage Passing for MA Late-‐Placement Student Teachers Fall 2006-‐Spring 2010

92.4 90.3 91.2 94.8

91.9

84.0 88.8

96.3

84.9 91.1

95.7 86.5

77.3

89.3 92.7

89.2 95.9

81.5 92.4

92.6 91.7

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.

Pedagogical Content Knowledge of Content

Long/Short Term Planning Constraints on Teaching & Learning

Criteria and Standards Feedback, Reflection and use for Planning

Teacher Interaction w/ Pupils Classroom Interaction

Functioning of Learning Groups Transitions

Materials and Supplies Mutual Expectations

Awareness of Pupil Behavior Lesson Structure and Time Management*

Clarity of Goals

Knowledge of Students Teacher/ Pupil Communications

Discussion Patterns Relationships w/ Adults

Cultural Context of School & Community Use in Future Teaching

% >= 3


37

performance by the Math field supervisors. The Math faculty has also pointed to the need for an assessment more aligned with their content area, or at least supplemented by intra-domain and content-specific dimensions. NYU faculty is currently engaged in conversations concerning the competing theories of the cross-disciplinary nature of pedagogical skill, which undergirds the Danielson approach, versus the discipline-specific approach advocated by NYU’s math education faculty, among others. A major study funded by the Bill and Melinda Gates Foundation, the Measurement of Effective Teaching Project, is currently assessing the validity of various frameworks – including Danielson (the most widely used) and several others that do incorporate disciplinary perspectives (Kane, Cantrell, et al., 2011). NYU faculty is eagerly awaiting the results of this study to inform our discussions about student teacher assessment.

TABLE 4.2.1b

Summary of performance on DRSTOS-R Total Scores for student teachers in their last placements by program certification areas, fall 2006 – spring 2010

Undergraduate

Program * N Assessed % >=3** M SD Dual Early Childhood 48 54.2% 3.13 0.58 Childhood/ Dual Childhood 101 66.3% 3.11 0.54 English 24 75.0% 3.14 0.56 Social Studies 22 77.3% 3.36 0.61 Math 20 30.0% 2.72 0.44 Music 6 50.0% 3.01 0.43

Graduate

Early Childhood/ Dual Early Childhood 44 72.7% 3.30 0.49 Childhood/ Dual Childhood 66 75.9% 3.27 0.45 Science 47 93.6% 3.63 0.37 English 38 78.9% 3.27 0.37 Social Studies 38 78.9% 3.35 0.59 Math 43 55.8% 3.06 0.35 MMS 97 80.5% 3.38 0.42 Educational Theatre 34 82.4% 3.22 0.45 Art 12 33.3% 2.97 0.37 Dance 7 100% 3.28 0.18 Music 6 50.0% 3.16 0.84

* The following programs are not reported because of low N (< 5): Undergraduate = Science, Special Education, and Educational Theatre Graduate = Special Education and Literacy ** Values in bold font meet the program standard of 80% >=3; values in bold italics fall within the 95% confidence interval around the standard, which means they are not significantly lower than the standard, p<.025. 4.2.2. New York State Teacher Certification Exams (NYSTCE) Scores Validity The comparative validity of both NYSTCE exams and GPAs as measures of the claims was explored through the analysis of correlations between and among these exams. Convergence was operationally defined as statistically significant correlations with at least a moderate effect size; and divergence was defined as a non-significant correlation and/or one with a small effect size. The ATS-W showed statistically significant medium correlations with Pedagogy Courses GPA, and small but significant correlations with the other GPAs for both BS and MA students, thereby supporting the validity for using both measures for Claim 2. The


38

patterns of correlation for the other exams and GPAs are more complex. The CST exams showed convergence with the Cross-cutting Themes GPA for both the BS and MA students, but divergence from the other GPAs, including the Content GPA. This suggests that the CSTs measure content knowledge differently than the Content GPA, perhaps the broader range of content knowledge and skills that is measured by the Cross-cutting Themes GPA. Finally, the LAST showed medium and significant correlations with the Cross-cutting Themes GPA for MA students (their total undergraduate GPA), and a more complex pattern of correlations for BS students. Taken together, these findings provide weak to moderate support for using the NYSTCE exams to assess the claims. Sample Size and Representativeness A match of the files of Steinhardt teacher education graduates for the classes of 2006 through 2010 to the NYSTCE test score files provided by the test publisher, National Evaluation Systems, yielded scores for the following numbers and percentages of teacher education graduates over the five-year period: for the LAST, 436 BS students ( 82.6% of the BS graduates) and 1,414 MA students (66.4%); for the ATS-W, 429 BS students (81.3%) and 1,508 MA students (70.8%); and for the CSTs, 409 BS students (77.5%) and 1,541 MA students (72.4%). In addition, graduates from all of the certification program areas were included in the sample. Therefore, the sample was large with respect to the population size and representative with respect to program areas. Overall Results of the Assessment Mean scaled scores and standard deviations and the percents of students passing the exams were computed. In order to assess the claims, two comparisons were applied to the data. First, the number of standard deviations by which the mean scaled scores exceeded the passing score, 220, was computed for each test and each group.9 These statistics were used as measures of the effect sizes of the Steinhardt scores in relation to the passing-score standards. The criterion for this measure was an effect size (ES) of at least 0.80, a large and educationally meaningful effect size. Second, the obtained percents passing each type of test were compared to the program standard of 90%. The results of the comparisons, presented in detail in Table 4.2.2, show that the BS and MA students in both classes passed the LAST, a measure of the Learning-to-Learn cross-cutting theme, with effect sizes that were nearly three times the criterion. Even stronger results were obtained for the ATS-W, a measure of Claim 2, Pedagogical Knowledge. The results for the CSTs, while not as strong as the previous two measures, also far exceeded the criteria, thereby providing strong support for Claim 1, Content Knowledge.

9 The pooled standard deviations for the Steinhardt graduates across multiple test administrations were used in these calculations.


39

TABLE 4.2.2

Mean Scaled scores, effect sizes, and passing rates for Steinhardt teacher education graduates on New York State Teacher Certification Exams (NYSTCE): Graduates from 2006 – 2010

Statistic Class of 2006 Class of 2007 Class of 2008 Class of 2009 Class of 2010 Totals BS MA BS MA BS MA BS MA BS MA BS MA

Liberal Arts and Sciences Test (LAST) Cross-Cutting Theme: Learning to Learn

N Tested 92 254 85 311 98 314 104 308 92 227 471 1414 Mean Scaled Score 268.9 270.5 267.7 267.5 267.3 268.5 269.1 270.6 269.6 270.7 268.5 269.5 Standard Deviation 15.3 18.0 16.4 17.3 19.0 17.5 31.1 16.1 18.6 17.5 21.1 17.3

SDs Above Passing * 3.19 2.81 2.91 2.75 2.49 2.77 1.58 3.14 2.66 2.90 2.57 2.86 Percent Passing ** 100% 99.6% 100.0% 99.0% 98.0% 99.7% 97.8% 100.0% 97.8% 98.7% 98.9% 99.4%

Assessment of Teaching Skills-Written (ATS-W) Claim 2: Pedagogical Knowledge N Tested 90 278 84 340 97 326 97 328 95 236 463 1508

Mean Scaled Score 267.0 268.0 267.1 269 270.1 268.2 272.7 268.2 270.8 270.4 269.7 268.7 Standard Deviation 13.4 14.8 14.8 15.5 13.8 15.8 13.3 13.5 14.5 13.9 14.1 14.8

SDs Above Passing * 3.51 3.24 3.17 3.16 3.64 3.05 3.95 3.57 3.51 3.63 3.56 3.29 Percent Passing ** 100% 99.6% 100.0% 99.7% 100.0% 99.7% 100% 100% 100% 100% 100% 99.8%

Content Specialty Tests (CSTs) *** Claim 1: Content Knowledge N Tested 81 316 81 352 90 322 92 310 97 241 441 1541

Mean Scaled Score 249.4 251.5 245.2 255.6 250.0 255.9 252.0 257.4 254.6 254.4 250.4 255.0 Standard Deviation 18.6 21.5 33.0 19.6 17.6 19.4 18.1 21.3 21.4 19.8 22.4 20.4

SDs Above Passing * 1.58 1.47 0.76 1.82 1.71 1.85 1.77 1.76 1.62 1.74 1.49 1.72 Percent Passing ** 92.6% 95.9% 93.8% 97.7% 97.8% 99.0% 98.9% 97.7% 95.9% 96.7% 95.9% 97.5%

* SDs Above Passing = Effect Size = (MSS - 220)/SD; the program standard is an ES >= .80, large and meaningful ** Passing score = 220 on a scale of 100 – 300 *** If a student has multiple tests, data are based on the most recent exam


40

Disaggregation by Program Area

The CST results were disaggregated by certification area (see Figures 4.2.2a for BS students and 4.2.2b for MA students,). Students took CSTs in all of the certification areas. The mean scores for all certification areas far exceeded the passing score of 220; however there were some noteworthy differences. Students taking the Math exam had the highest means for both the BS group (269.9) and the MA group (277.1). For the BS group, mean scores were in the 250 range for the Elementary Education CST, Educational Theatre, English, Science, and Music. For the MA group, the mean for Literacy was 268.2, and means ranged in the 250’s for Elementary Education CST, Science, English, Educational Theatre, ESOL, and Social Studies. The mean scaled scores exceeded the program standard for all program certification areas for both BS and MA students. For BS students, the ES ranged from 2.23 for Math to 1.14 for (Teachers of) Students with Disabilities; for MA students, the ES ranged from 2.80 for Math to 1.02 for Dance

Figure 4.2.2a Mean scaled scores for most-frequently taken NYSTCE Content Specialty Tests Note. Data are based on the most recent CST exam taken by each graduate. Graduates who took multiple CSTs are counted in only one subject.


41

4.2.3. Student Teacher End-of-Term Feedback Questionnaire (ETFQ) Sample Representativeness A total of 1,311 program completers, 530 BS and 781 MA graduates, completed ETFQs at the end of their last student teaching placement for the Classes of 2006-10. This represents an overall response rate of 50%. The respondents represented all of the program certification areas, with the exception of the BS programs in music and special education and the MA programs in music, art, and special education. Reliability and Validity Reliability was assessed through analysis of the stability of results. The mean scores for the three claims showed high consistency across the four cohorts (see Table 4.2.3). The size of the differences between each cohort mean and the grand mean was calculated in sigma units, using the SDs for the grand means. For BS students, the largest differences were 0.28 for Claims 2 and 3 and 0.21 for Claim 1, all for the Class of 2008, when the means fell just below the program standard. For MA students, all mean differences were less than 0.20, signifying that they were not meaningful. Results of the Assessment The participants completed the survey online responding to questions about the extent to which mentoring by their supervisors and cooperating teachers had enhanced their knowledge and skill in the areas of Content Knowledge (Claim 1), Pedagogical Knowledge (Claim 2), and Teaching Skill (Claim 3, Clinical Knowledge). Table 4.2.3 displays the mean scores on a 5-point scale ranging from Very Poorly (1) to Very Well (5) for each class of BS and MA student teachers and the totals across all five classes. The target standard was a total mean of 4.0 (Well) on the items for each claim. The target means were attained on the totals for both BS and MA

Figure 4.2.2b Mean scaled scores for most frequently-taken NYSTCE Content Specialty Tests


42

groups for all three claims. There was little variability in the means across the five classes, although the highest means were observed for the Class of 2010. Also, there was little variability in the means for program certification areas, with the exception of very high ratings for students in the literacy MA program on all three measures, although the n was small. The consistency of strong student teacher perceptions of the impact of mentoring on their content area and teaching knowledge and skills lends support to Claims 1 – 3.

TABLE 4.2.3 Mean Scores on the Claim Scales on the End of Term Feedback Questionnaire for Steinhardt students in

their final student teaching placement: Classes of 2006 – 2010 Class of 2006 Class of 2007 Class of 2008 Class of 2009 Class of 2010 Totals *

BS MA BS MA BS MA BS MA BS MA BS MA Claim 1 Scale: Content Knowledge

N 150 192 150 215 98 156 42 59 89 158 529 780 Mean 3.93 3.96 4.11 4.03 3.84 4.01 3.95 3.94 4.18 4.07 4.01 4.01

SD 0.84 0.86 0.75 0.81 0.83 0.88 0.8 1.13 0.86 0.91 0.81 0.88 Claim 2 Scale: Pedagogical Knowledge

N 150 192 151 216 98 156 42 59 89 158 530 781 Mean 4.04 4.07 4.26 4.17 3.92 4.16 4.18 4.13 4.29 4.27 4.13 4.16

SD 0.82 0.79 0.68 0.76 0.86 0.79 0.61 0.91 0.72 0.78 0.75 0.79 Claim 3 Scale: Clinical Knowledge

N 150 192 151 216 98 156 42 59 89 158 530 781 Mean 4.09 4.06 4.22 4.18 3.93 4.16 4.21 4.11 4.27 4.25 4.14 4.16

SD 0.71 0.8 0.7 0.72 0.85 0.8 0.63 94 0.78 0.76 0.74 7.81 Notes: Claim 1 scale items are Items 9 and 18; Claim 2 scale items are Items 7 and 15; and Claim 3 scale items are Items 8, 11, 16, and 19. Items are measured on a 5-point Likert scale with scale values of (1) “Very Poorly”, (2) “Poorly”, (3) “Average”, (4) “Well”, and (5) “Very Well”.

* Total means in bold meet the program standard of 4.0. 4.2.4. Educational Beliefs Multicultural Attitudes Survey (EBMAS) Sample Representativeness The EBMAS sample used to assess the claims included 175 students, 57 BS and 118 MA who were completing their programs at the time of the assessment in spring 2010. The sample represents 64.0% of the BS graduates and 57.0% of the MA graduates for spring 2010, for an overall response rate of 59.1%. The sample included students from every program certification area. Reliability and Validity Reliability: EBMAS’ four scales showed moderate to strong internal consistency reliability with Cronbach’s alpha coefficients as follows: MA/SJ Alpha =.887, n=659; GTE Alpha =.674, n=671; PTE1 Alpha =.749, n=623; and PTE2 Alpha=.739. n=663.

Construct validity: First, structural validity was examined through exploratory factor analysis of the full database (N = 664), using a principal components model with varimax rotation. The scree plot revealed diminishing returns after a four-factor solution that accounted for 48% of the item variance. The rotated components matrix was consistent with the item-factor alignment that guided the development of EBMAS with several exceptions. EBMAS was


43

constructed with items from separate scales, including one that measured caring/social justice and another that measured multicultural attitudes. It was hypothesized that separate factors would emerge for each; however factor analysis found that a single factor explained the items from the separate scales. Accordingly, these items were merged into a single scale that was named Multicultural Awareness/Social Justice (MA/SJ). Next, items that were drawn from the TES to measure a single construct called personal teacher efficacy broke into two separate factors: one, which was labeled Personal Teacher Efficacy 1 (PTE 1), was comprised of items that measured the student teachers’ belief in their ability to deal with specific student problems; the second, which was labeled PTE 2, measured their belief that they would be able to help students succeed. The other hypothesized factor, General Teacher Efficacy (GTE), emerged as expected. The 28 items with high loadings on one of the four factors were organized into four scales by summing the values of responses, with negatively-stated items appropriately flipped for scoring.

Construct validity was further supported by a comparison of the relationships between mean scale scores and the levels of program experience of students in the full database. As hypothesized, the mean scores for high-credit BS students were statistically significantly higher than those for the medium- and low-credit students, thereby demonstrating that the scale scores were influenced by time in the program. Results from a similar comparison for MA students were not as strong. The mean scores for high-credit MA students were statistically significantly higher than low credit MA students for only one scale: PTE 1, Student Problem Solving. The smaller differences for MA students may be attributed to smaller differences between the credit groups on time in program than for the BS students.

Results of the Assessment

The program standard was set at mean scores of 4.5 on all four scales for BS and MA students in their final semester. This value is half-way between neutral and the highest possible score, thereby representing high levels of beliefs in teaching efficacy and social justice/multiculturalism. The data were analyzed for the total BS and MA students; they were not disaggregated by certification area because of the low sample size. Table 4.2.4 displays the mean scale scores for these students and performance against the program standard. The means for both BS and MA students met the standard for three of the four scales. Both groups met the standard for MA/SJ and GTE; BS students met the standard for PTE1 while MA students met the standard for PTE2. Although the mean for the latter was 4.39 (SD=0.69), the program standard was within the 95 percent confidence interval around the obtained value, thereby indicating that the difference might be due to sampling error. The results provide evidence in support of the cross-cutting theme of Multicultural Perspective, and Claim 4, Caring Professionals; they also provide partial support for Claim 3, Competence in Clinical Knowledge.


44

TABLE 4.2.4

Mean scores and performance against program standards on the EBMAS for BS and MA Steinhardt teacher education program completers in the Class of 2010

Scale (Claim) BS Program Completers MA Program Completers N Mean SD M-4.50* Performance N Mean SD M-4.50* Perform.

Multi-cultural Attitudes/Social Justice (Claim 4 & CCT Multicultural Perspective)

57 5.45 0.47 0.95 Met program standard 117 5.19 0.60 0.69

Met program standard

General Teacher Efficacy (Claim 4) 57 5.24 0.65 0.74 Met program

standard 118 4.83 0.84 0.33 Met

program standard

Personal Teacher Efficacy 1: Student problem solving (Claim 3)

57 4.49 0.83 -0.01 Met program standard 118 4.16 0.78 -0.34

Below program standard

Personal Teacher Efficacy 2: Student success (Claim 3)

56 4.20 0.79 -0.30 Below

program standard

118 4.39 0.69 -0.11 Standard within CI **

Responses are measured on a 6-point scale of agreement as follows: (1) Strongly Disagree (2) Moderately Disagree (3) Slightly Disagree (4) Slightly Agree (5) Moderately Agree (6) Strongly Agree. * The program goals are means of 4.50 (negative items were reverse coded). ** The program standard is within the 95% confidence interval around the observed mean.

4.2.5. Grade Point Averages (GPA) Reliability and Validity Reliability was assessed through analysis of the stability of the GPA results across cohorts. Inspection of Tables 4.2.5a and 4.2.5b finds a high degree of consistency in the GPAs across the five classes 2006 – 2010. All of the standardized differences between each class’ GPAs and the grand mean, with the sole exception of the Teaching Skills (TS) for BS graduates, are less than 0.20 standard deviations, indicating that the differences are not meaningful. The TS for 2010 graduates shows a large difference from the grand mean on the positive side, while the TS for 2006 shows a moderate difference on the low side. These differences are not sufficient to threaten inferences about the claims that are based on the grand means.

The empirical validity for the GPA measures was investigated through correlations with the NYSTCE exams. As can be seen on Table 4.2.2 and discussed in section 4.2.2 of the results above, the pattern of correlations provides weak to moderate support for the use of the Content Knowledge GPA (CK) to assess Claim 1, the Pedagogical GPA (PK) to measure Claim 2, the Teaching Skills GPA (TS) to measure Claim 3, and the Cross-Cutting Themes GPA (CCT) to


45

measure the cross-cutting themes claims, especially Learning to Learn. It should be noted that the CCT for BS students includes courses taken at CAS and Steinhardt; for MA students, it is the undergraduate GPA.

Assessment of the Claims Overall results: Tables 4.2.5a and 4.2.5b display the four types of mean GPAs for BS and MA students for the five classes in the inquiry, along with the overall means. The tables also show the alignment between the claims and the GPAs. Given the high stability of GPAs across years, the analysis focuses on the total or grand means. Seven of the eight grand means exceed the program criterion of 3.00; the lone exception is the CCT for BS students, which is 2.79.

TABLE 4.2.5a Mean GPAs of NYU BS teacher education graduates by claims (Classes of 2006

- 2010) Claim GPA* Statistic 2006 2007 2008 2009 2010 Total**

1 CK

Mean 3.15 3.23 3.17 3.1 3.09 3.15 SD 0.61 0.73 0.67 0.75 0.62 0.68 N 105 104 108 125 118 560

2 PK

Mean 3.74 3.72 3.73 3.68 3.63 3.7 SD 0.26 0.33 0.31 0.26 0.25 0.28 N 105 105 109 127 120 566

3 TS

Mean 3.23 3.28 3.37 3.42 3.55 3.37 SD 0.25 0.24 0.28 0.28 0.26 0.29 N 105 105 109 127 120 566

Cross-cutting CCT

Mean 2.76 2.78 2.88 2.82 2.69 2.79 SD 0.97 1.08 0.97 1.09 1.05 1.04 N 105 105 109 126 120 565

* Types of GPA: CK=Content Knowledge; PK=Pedagogical Knowledge; TS=Teaching Skill; CCT=Cross Cutting Themes

** Total means in bold font meet or exceed the program standard of 3.0

TABLE 4.2.5b Mean GPAs of NYU MA teacher education graduates by claims (Classes of 2006

- 2010) Claim GPA* Statistic 2006 2007 2008 2009 2010 Total**

1 CK Mean 3.73 3.72 3.59 3.73 3.67 3.69 SD 0.56 0.74 0.89 0.69 0.78 0.75 N 174 294 286 319 260 1333

2 PK Mean 3.86 3.87 3.86 3.87 3.85 3.86 SD 0.17 0.18 0.26 0.16 0.21 0.2 N 271 480 435 443 330 1959

3 TS Mean 3.75 3.88 3.88 3.88 3.87 3.86 SD 0.35 0.25 0.29 0.23 0.34 0.29 N 230 422 390 389 312 1743

Cross-cutting

CCT ***

Mean 3.27 3.31 3.34 3.32 3.36 3.32 SD 0.4 0.37 0.36 0.36 0.35 0.37 N 314 361 308 311 245 1539

* Types of GPA: CK=Content Knowledge; PK=Pedagogical Knowledge; TS=Teaching Skill; CCT=Cross Cutting Themes


46

** Total means in bold font meet or exceed the program standard of 3.0 *** CCT for MA students is based on total undergraduate GPA

Disaggregation by program area: To gain a better understanding of the dynamics underlying the below-standard performance in CCT GPA by the BS graduates, the data were disaggregated by certification program (see Table 4.2.5c. Mean CCTs were above the standard for two program areas, Science Education and Math Education. The means were just below the standard for English Education, Social Studies Education, and Educational Theatre. The means for the remaining programs were well below the standard. In summary, the GPA data support the claims of Content Knowledge, Pedagogical and Content Knowledge, and Clinical Competence. They provide partial evidence for the cross-cutting themes, especially Learning to Learn, supporting the claim for MA students and BS students in Math and Science.

TABLE 4.2.5c Mean CCT GPA by certification area program(Classes of 2006-2010)

Certification area program Mean CCT* N SD

Science Ed. 3.15 10 0.53 Math Ed. 3.08 49 0.90

English Ed. 2.94 86 0.92 Social Studies Ed. 2.93 51 1.13

Ed. Theatre 2.88 50 0.70 Music Ed. 2.66 43 0.78

Dual Childhood Ed. 2.61 168 1.13 MMS 2.60 17 1.39

Dual Early Childhood Ed. 2.59 90 1.14 Total 2.76 564 1.04

*The program standard is a mean GPA of 3.0

4.2.6. Program Exit Survey Sample Representativeness There was evidence that the 232 (38.4%) survey respondents were representative of the population of program graduates for May 2009 and 2010. The split for degree program was 21% BS and 79% MA, under-representing the former compared to the population split of 30.8% and 69.2%, respectively, but maintaining the balance between the two. In addition, the respondents represented every certification area program, except Art Education. Reliability and Validity To assess the stability of the results, z-tests for the difference between proportions were applied to the percents of “Very Well” and “Moderately Well” responses for each item of the 2009 and 2010 surveys. None of the differences in responses between the two years was statistically significant. Inspection of the rank order of items for percent of “Very Well” and “Moderately Well” responses showed a close correspondence between both the NYU BS and MA samples and the Levine sample. The NYU rankings for all 11 items common to the Levine and NYU surveys were within two ranks of one another, with one exception; both BS and MA


47

NYU samples ranked their preparation in “Address the needs of students from diverse cultures” higher than the Levine sample; the item ranked third out of 11 for the NYU samples versus eighth out of 11 for the Levine sample. The convergence of data supports the substantive validity of the Program Exit Survey as a measure of self-perceived preparation to teach and, by extension, its validity for making inferences about the claims. Assessment of Claims

Overall results: Table 4.2.6 displays the numbers and percents of surveyed students who indicated that they were “Very Well” or “Moderately Well” prepared to begin teaching in each of 15 skill and knowledge areas, as well as the alignment between items and program claims. Data are shown for BS and MA students for each year and combined. The percents responding well or very well on the two-year totals are compared to the program standard of 80 percent, which is a level that exceeds the percents obtained in the Levine study by six to 53 percent, depending on item. This ambitious standard reflects Steinhardt’s commitment to develop teachers who have the knowledge, skills, and dispositions that will ensure they are highly effective in promoting the learning of their pupils. As seen in Table 4.2.6, BS students met the 80 percent standard for 12 of the 15 skills (three within the 95% confidence interval), while the MA students met the standards in nine (one within the 95% confidence interval). In general, BS students expressed more confidence in their preparation for teaching than MA students. Both groups met the program standard for the two items on Content Knowledge and four of the six items on Pedagogical Knowledge. BS students met standards on the two Clinical Skill items while MA students met one; only 67.7% of the MA students felt prepared to “maintain order and discipline in the classroom.” BS students met the standard on the three Caring Professionals items while MA students met one. Finally, while both groups met standard on the item related to the cross-cutting Multicultural theme, both fell below standard on the item related to integrating technology into teaching. Disaggregation by program area: In order to determine the extent to which responses varied by certification area, the data were disaggregated by program. To reduce the threat to validity from sampling bias, the responses of BS and MA students were combined for this analysis. It should be noted that the numbers of respondents varied among program areas, and that there were no respondents from the Dual Early Childhood/Special Education, Dual Childhood/Special Education, or Art Education program areas. The matrix of item responses (Percent “Very Well” or “Moderately Well”) by program certification areas was inspected in order to identify noteworthy variations, using a criterion of plus or minus 15% to form the value for the total sample. The seven respondents for Dance Education expressed extremely high perceptions of their preparation; 100% felt prepared to teach on 10 of 14 items (they did not respond to the item on technology); the one item in which a low percentage (14.3% or one student) felt prepared was “Work with parents.” Other programs with higher than average levels of perceived preparation for at least four items included Multicultural/Multilingual Studies (MMS), English Education, and Early Childhood Education (MA program), although the latter also had several items with low levels of perceived preparation. The Science and Social Studies Education programs had low levels of perceived preparation on multiple items, although the sample size was too small to eliminate the possibility


48

of sampling bias. The program-specific results have been shared with program faculty and are being used to help inform discussions aimed at program improvement. TABLE 4.2.6 Numbers and percents of Steinhardt teacher-education program completers who reported on the Program Exit Survey that their programs prepared them very or moderately well to begin teaching: Classes of 2009 and 2010

Claim How well did your teacher education program prepare you to:

Responded Very Well or Moderately Well Combined Class of 2009 Class of 2010 Totals * BS MA BS MA BS MA

Content knowledge Have a mastery of your subject area N 19 72 18 61 37 133

% 76.0% 81.8% 85.7% 81.3% 80.4% 81.6% Content knowledge

Implement state/district curriculum & standards

N 22 71 18 58 40 129 % 88.0% 81.6% 85.7% 78.4% 86.9% 80.1%

Pedagogical knowledge Understand how students learn N 23 82 20 67 43 149

% 92.0% 94.3% 95.2% 88.2% 93.5% 91.5% Pedagogical knowledge

Use different pedagogical approaches

N 24 83 20 69 44 152 % 96.0% 94.3% 95.2% 89.6% 95.6% 92.1%

Pedagogical knowledge

Use student performance assessment techniques

N 21 72 18 62 39 134 % 84.0% 82.7% 85.7% 82.7% 84.8% 82.7%


Address needs of students with disabilities

N 20 67 15 57 35 124 % 80.0% 76.1% 71.4% 76.0% 76.1% 76.1%


Address needs of students with limited English proficiency

N 14 55 11 51 25 106 % 58.4% 61.5% 52.4% 67.1% 55.6% 64.1%

Pedagogical knowledge Work with parents N 17 40 17 40 34 80

% 68.0% 45.0% 68.0% 45.0% 68.0% 45.0%

Clinical skill Maintain order & discipline in the classroom

N 23 58 17 54 40 112 % 92.0% 65.6% 81.0% 70.1% 87.0% 67.7%

Clinical skill Impact my students' ability to learn N 22 83 21 67 43 150 % 95.7% 95.4% 100.0% 89.3% 97.8% 92.6%

Caring Professionals

Work collaboratively with teachers, administrators and other school personnel

N 20 72 16 61 36 133

% 80.0% 82.5% 76.2% 81.3%

78.3% 81.9% Caring Professionals

Identify & use resources within the community where you teach

N 20 59 18 58 38 117 % 80.0% 67.8% 85.7% 76.3% 82.6% 71.8%

Caring Professionals

Participate as a stakeholder in the community where you teach

N 20 55 16 50 36 105 % 80.0% 63.9% 76.2% 65.8% 78.3% 64.8%

Cross-cutting theme

Address needs of students from diverse cultures

N %

22 80.0%

79 89.8%

20 95.2%

66 86.8%

42 91.3%

145 88.4%

Cross-cutting theme Integrate technology into teaching N 14 47 14 47

% 66.7% 61.8% 66.7% 61.8% * Total percents in bold meet or exceed the program criterion of 80%; those in bold italics have the program criterion within the 95% confidence interval for the observed value.

Responses recorded on a four-point scale as follows: (4) Very Well (3) Moderately Well (2) Somewhat Well (4) Not Well at


49

All. Most items were taken from Arthur Levine's survey of teacher education graduates (2006)

4.2.7. One-Year Follow-Up Survey Sample Representativeness The program characteristics of the respondents were similar to those of the target population of graduates but there were some differences. The sample of 322 respondents was split 32.6% BS graduates and 67.4% MA graduates. Compared to the split of 27.9% BS and 72.1% MA graduates for the full population of graduates from the classes of 2007 – 2009, the survey sample had a small but statistically significant under-representation of BS graduates. All of the certification areas were represented among the respondents although there were some differences in their relative proportions. Most notably, Educational Theatre graduates were over-represented in the sample of respondents. Overall, the sample of respondents does not show bias in measured characteristics that would pose a threat to the validity of inferences concerning the claims. Reliability and Validity To assess the stability of the results, z-tests for the difference between proportions were applied to the differences in means across the three administrations for the 15 survey items. Only three of the 45 tests were statistically significant. Therefore, the overall survey results showed a high degree of stability/reliability across the three years of data. Inspection of the rank order of items for percent of “Well” and “Moderately Well” responses showed a close correspondence between both the NYU BS and MA samples and the Levine sample, similar to the same analysis for the Program Exit Survey. The NYU rankings for all 11 items common to the Levine and NYU surveys were within two ranks of one another, with one exception; the NYU sample ranked their preparation in “Address the needs of students from diverse cultures” much higher than the Levine sample - 3rd out of 11 for the former two samples versus 8th out of 11 for the latter—the same rankings as observed for the Program Exit Survey. The convergence of data supports the substantive validity of the One-Year Follow-Up Survey as a measure of self-perceived preparation to teach and, by extension, its validity for making inferences about the claims. Assessment of the Claims Overall results: Table 4.2.7 displays the numbers and percents of responding graduates who reported they felt “Very Well” or “Moderately Well” prepared to teach by the Steinhardt teacher education program in 15 skill areas. Unlike the data for the Program Exit Survey, these results are for BS and MA students combined. The decision to combine the data was based on the similarity of results for BS and MA graduates of the Program Exit Survey and an effort to generate more robust parameter estimates. The survey results met the program standard of 80% for 6 of the 15 items: 1 of the 2 Content Knowledge items; 3 of the 6 Pedagogical Knowledge items; 1 of the 2 Clinical Skill items; none of the 3 Caring Professional items; and the cross-cutting multicultural theme item but not the cross-cutting integrating technology item. The profile of results, as determined by the rank order of items, is the same as observed for the Program Exit Survey. However, the overall percents expressing feelings of high and moderate preparation are, with a few exceptions, 5 – 10 percentage points lower in the One-Year Follow Up Survey. This may be attributable to the harsh realities of schools and classrooms as perceived by new teachers during the induction year. This appears to be especially true for those


50

who teach in urban schools, as indicated by a statistically significant effect for these schools on survey responses (Tobias and Tian, 2010). TABLE 4.2.7 - Numbers and percents of Steinhardt teacher-education graduates who reported on the One-Year Follow-Up Survey that their programs had prepared them very or moderately well to begin teaching: Classes of 2007 – 2009

Claim How well did your program prepare you to: Responded very or moderately well combined Graduating class: 2007 2008 2009 Totals *

Content knowledge Have a mastery of your subject area N 68 67 107 242

% 71.6% 78.8% 79.9% 77.1% Content knowledge

Implement state/district curriculum & standards

N 59 60 102 221 % 62.1% 70.6% 76.1% 70.4%

Pedagogical knowledge Understand how students learn N 79 74 117 270

% 83.2% 87.0% 86.7% 85.7% Pedagogical knowledge Use different pedagogical approaches N 76 75 123 274

% 80.0% 88.2% 92.5% 87.5% Pedagogical knowledge

Use student performance assessment techniques

N 64 66 111 241 % 67.3% 77.7% 82.8% 76.7%

Pedagogical knowledge Address needs of students with disabilities N 56 60 79 195

% 58.9% 70.6% 59.0% 62.1% Pedagogical knowledge

Address needs of students with limited English proficiency

N 36 47 63 146 % 37.9% 55.3% 46.7% 46.4%

Pedagogical knowledge Work with parents N 42 48 59 149

% 44.2% 56.4% 42.1% 46.5%

Clinical skill Maintain order & discipline in the classroom

N 68 49 84 201 % 71.6% 57.7% 62.7% 64.0%

Clinical skill Impact my students' ability to learn N 73 72 116 261 % 76.8% 85.7% 86.6% 83.4%

Caring Profs. Work collaboratively with teachers, administrators & personnel

N 65 61 102 228 % 68.4% 71.7% 75.6% 72.4%

Caring Profs. Identify & use resources within the community where you teach

N 54 61 93 208 % 56.8% 71.7% 68.9% 66.0%

Caring Profs. Participate as a stakeholder in the community where you teach

N 40 55 76 171 % 42.1% 64.7% 56.3% 54.3%

Cross-cutting theme

Address needs of students from diverse cultures

N %

70 73.7%

67 78.9%

111 82.2%

248 78.7%

Cross-cutting theme

Integrate technology into the grade level or subject taught

N 53 51 90 194 % 55.8% 59.2% 66.7% 61.4%

* Total percents in bold meet or exceed the program criterion of 80%; those in bold italics have the program criterion with in the 95% confidence interval for the observed value.

Responses recorded on a four-point scale as follows: (4) Very Well (3) Moderately Well (2) Somewhat Well (4) Not Well at All. Most items were taken from Arthur Levine's survey of teacher education graduates (2006)


51

Disaggregation by program area: Using the same analytic model and with the same cautions as described above for the Program Exit Survey, disaggregated data were explored for differences among program certification areas. There were respondents for every program, although the frequencies varied widely. As they did in the Program Exit Survey, the respondents from Dance Education reported high degrees of preparation to teach one year after graduation. As for the respondents overall, their perceptions were somewhat muted compared to the Program Exit Survey, but still mostly positive. High percents felt prepared in six of the 15 teaching skills, and low percents on only one, namely “Work with parents” (low also in the Program Exit Survey). Respondents from some of the other programs claimed an even broader scope of preparation, as indicated by high percents feeling prepared over a larger number of skills. These programs included Science Education (which had low levels of perceived preparation on the Program Exit Survey), Dual Early Childhood, and Educational Theatre. Programs with low perceptions of preparation on several skill areas included Art Education, Social Studies Education, and Special Education, excluding the dual certification programs. The caveat about the possibility of sampling bias due to low n applies here. The program-specific results for both surveys have been shared with program area faculty and are being used to help inform discussions aimed at program area improvement. We expect to see changes over the next 2 years and will track these in the annual TEAC reports. 4.2.8. Graduate Tracking Study Reliability Analyses of the tracking data obtained from the NYCDOE for the years 2004 – 2009, showed patterns of employment that were systematically consistent with identifiable factors that have affected NYU enrollments and the market for teachers in NYC, thereby supporting the reliability of the data. Results of the Assessment Types of NYC schools in which graduates are employed: Consistent with NYU’s mission to be in and of the city, a key focus of Steinhardt’s teacher education programs is to prepare its graduates to teach in NYC public schools. Moreover, true to NYU’s claim of developing caring professionals who are concerned with social justice, competent interculturally, and attracted to meeting the needs of economically disadvantaged students living in cities, the goal is for graduates to teach in schools demographically similar to NYC’s public schools. To assess the extent to which this goal has been realized, the graduate tracking study collects data on the demographics of the NYC public schools in which Steinhardt graduates teach. Demographic data on the schools of graduates from the classes of 2004 – 2008 were downloaded from the NYCDOE school-wide Annual School Progress Report file for the 2007-08 school year. The standard set by faculty was no statistically significant differences in high-need demographics for the graduates’ schools compared to the NYC public schools overall. The mean percentages of students in four high-need, at-risk categories were calculated for the graduates’ schools and statistically compared to the respective means for all NYC schools within the same school type (i.e., grade levels) using Z tests for differences in proportions (see Table 4.2.8a). The results indicate that for the most part, the high-need demographic characteristics of the graduates’ schools were not different statistically from NYC schools of the same type overall. There were no significant differences statistically between the graduates’ schools and all


52

NYC public schools in percent English language learners (ELLs) and percent special education for all five school types. There were also no statistically significant differences in percent Black and Hispanic and percent Title 1 eligible schools for graduates teaching in K – 8 schools and Transfer Schools for at-risk students. However, the mean percent Black and Hispanic for graduates’ schools were significantly lower in K-6, middle, and high schools, and the percents Title 1 eligible were significantly lower in graduates’ K-6 and middle schools. However, even where differences were observed, the mean percents of the graduates’ schools showed high levels of need. Thus, overall, these data support Claim 4, Caring Professionals, and the cross-cutting theme of multiculturalism. TABLE 4.2.8a

Comparison of the demographics of NYC schools in which NYU graduates first taught and all NYC schools disaggregated by school type (Sep. 2004 - Sep. 2008 grads)

Grads Schools All NYC Schools Diff. in

School Type Demographic N

Grads Mean SD Mean SD Means* Elementary %ELL 391 16.2 12.9 16.9 13.1 -0.7 %Spec. Ed. 391 19.4 8.7 16.9 6.3 2.5 % Black & Hispanic 391 61.7 31.9 70.8 31.3 -9 % Title 1 Eligible 391 59.2 28.8 68.7 23 -9.5 N Enrolled 391 636.3 251.6 639.3 277.9 -3 Middle %ELL 228 10.5 10.3 11.1 12.2 -0.6 %Spec. Ed. 228 18 7.2 16.6 7.4 1.4 % Black & Hispanic 228 73.4 28.1 81 25.1 -7.6 % Title 1 Eligible 228 62.8 22.7 68.9 19.3 -6.1 N Enrolled 228 651.5 387.5 584.6 419.2 66.9 K-8 %ELL 82 13.8 7.9 11.6 11 2.2 %Spec. Ed. 82 19.8 6.2 16.6 6.7 3.2 % Black & Hispanic 82 73.7 28.2 78.3 27.4 -4.6 % Title 1 Eligible 82 69.4 20.1 67.7 21.9 1.7 N Enrolled 82 672.9 289.1 684.6 290.8 -11.7 High School %ELL 304 10.7 17.8 12.6 18.5 -1.9 %Spec. Ed. 304 11.7 6.1 12.8 6.8 -1.1 % Black & Hispanic 304 72.9 26.1 82.3 22.1 -9.4 % Title 1 Eligible 304 54.9 22.6 61.4 19.9 -6.4 N Enrolled 304 1029.7 1128.5 898.3 1027.2 131.4 Transfer School %ELL 19 17.2 29.4 9.2 19.3 8 %Spec. Ed. 19 8.1 4.4 9.7 4.9 -1.7 % Black & Hispanic 19 75 25.6 86.7 17.6 -11.7 % Title 1 Eligible 19 53.8 21.4 63.4 15.5 -9.6 N Enrolled 19 510.4 293.4 317.8 229.2 192.6

Note 1: School demographic data were missing for 96 (8.6%) of the graduates who were working in NYC public schools * Differences in bold italics are statistically significant at p < .05. The program standard is that the


53

means for the percent of at-risk students in the schools of NYU graduates will equal to or higher than the means for all NYC public schools. The standard does not apply to enrollment.

Retention of NYU graduates in NYCPS: The NYCDOE data file also contained

information on the employment status of Steinhardt graduates in January, 2009. The status categories were “active in NYC schools,” “on leave (but not terminated),” and “no longer teaching in NYC public schools.” Juxtaposing these data with the dates of first employment in NYC schools for each cohort, it was possible to calculate the length of retention for graduates in

TABLE 4.2.8b

Retention status and years of teaching for Steinhardt graduates who began teaching in the New York City public schools within one year of graduation (Classes of 2004 -

2008, including Sept. 2008 graduates) Status on Jan. 1 2009

Total

Degree Class (Active status in NYC schools)

Active * On

Leave

No Longer

Teaching in NYC

BS

2004 (Teaching year 5 in NYCDOE)

N 20 1 10 31 % 64.5% 3.2% 32.3% 100.0%


N 40 2 9 51 % 78.4% 3.9% 17.6% 100.0%


N 34 0 8 42 % 81.0% 0.0% 19.0% 100.0%


N 25 0 9 34 % 73.5% 0.0% 26.5% 100.0%


N 28 NS 0 0 28 % 100.0% 0.0% 0.0% 100.0%

Total N 147 3 36 186 % 79.0% 1.6% 19.4% 100.0%

MA


N 69 9 48 126

% 54.8% 7.1% 38.1% 100.0% 2005 (Teaching year 4 in NYCDOE)

N 109 12 52 173 % 63.0% 6.9% 30.1% 100.0%


N 130 5 56 191 % 68.1% 2.6% 29.3% 100.0%


N 179 NS 1 32 212 % 84.4% 0.5% 15.1% 100.0%


N 209 1 10 220 % 95.0% 0.5% 4.5% 100.0%

Total N 696 28 198 922 % 75.5% 3.0% 21.5% 100.0%

Source: NYCDOE Human Resources Data Systems * Program standards for Active retention are as follows: first year = no standard (NS); year 2 = 80%; year 3 = 70%; year 4 = 65%; year 5 = 60%. The standards are set at 5 – 10%


54

above the estimated teacher retention rates for NYC public schools published in a staff report published by the NY City Council (July 2009).

each class. For instance, graduates in the Class of 2004 who were active in January 2009 would have been employed in the NYC public schools for five years. Those in the Class of 2005 who were still active would have been employed for four years, and so on. Using a report from the Investigation Division of the New York City Council (July 2009), Steinhardt faculty established standards for the retention of its graduates that were 5 – 10% above the estimated retention rates for NYC public school teachers overall (see footnote to Table 4.2.8b). Inspection of Table 4.2.8b reveals that both BS and MA graduates in all four classes met the respective retention standards, or had the standard within the 95% confidence interval, for their years of service in the NYC public schools. 4.2.9. Graduates’ Value-Added Effects on Pupils’ Standardized Test Scores Reliability and Validity Using a VAM dataset for NYU graduates provided by the NYCDOE, NYU researchers ran correlations of the calculated VAM effects for consecutive years for samples ranging in size from 45 – 60 graduates (Tobias, et. al., 2010). The Pearson product-moment correlations for VAM values from consecutive years were in the small but meaningful range for English language arts (ELA) (.18 and .23) and the medium and meaningful range for math (.34 and .35). These findings provide some support for the stability of VAM effects. Successive studies by NYU researchers using two different datasets provided by the NYCDOE, found moderately strong linear relationships between VAM effects and total years of teaching experience for NYU graduates in both ELA and math (Hummel-Rossi, et. al., 2009; Tobias, et. al., 2010). These findings supported the substantive validity of VAM as a measure of the graduates’ teaching skill, Claim 3. Results of the Assessment The VAM statistical model computes an expected score in performance levels ranging from 1 – 4.5 for each pupil based on student demographics and baseline test performance, teacher characteristics, and class demographics. The teachers’ VAM effect is the mean difference between their pupils’ actual and predicted performance. A VAM effect of zero indicates that the mean 2008 test scores of the teachers’ pupils are exactly as predicted by the VAM model; scores above zero indicate that mean performance is above prediction and scores below zero indicate actual performance that is below predicted. Table 4.2.9 displays the VAM data for Steinhardt graduates in ELA and math, disaggregated by years of teaching experience. For each experience group, the table shows mean actual and predicted performance and the difference between the two (the VAM effect), as well as the median percentile of the Steinhardt students compared to all NYC teachers of similar experience with the effect size, a measure of the strength of the graduates’ effects on their pupils. For ELA, the VAM effects for NYU graduates were positive for all experience categories and the total participants; four of the six groups and the total participants had median percentiles that were above 50. The program standard was set at an effect size of 0.20, which is interpreted as a small but meaningful effect (Cohen, 1996). For ELA, the program standard was met for four out of six experience groups and the total participants, thereby providing support for Claim 3, Competent in Clinical Knowledge. For math, the VAM effects were positive for five of the six experience groups and


55

for total participants; the median percentiles were above 50 for three experience groups and total participants. The effect sizes met the program criterion of .20 for only two experience groups; the effect size for total participants was 0.17. Therefore, the VAM evidence in math provides only weak support for Claim 3.

TABLE 4.2.9 Mean Actual-Versus-Predicted ELA and Math Test Gains for Pupils of Teacher Education Graduates

and Their Percentile Rank Among All District Teachers With Similar Years of Experience (NYU graduates teaching ELA and/or Math in grades 4 – 8 during 2008)

English language arts Mathematics

Yrs.of Exp. Statistic

Mean Prof. Level* Median %ile**

Effect Size ***

Mean Prof. Level* Median %ile**

Effect Size*** Actual Predict. Diff. Actual Predict. Diff.

< 1 M/Mdn 0.06 0.03 0.03 60 0.25 0.04 0.02 0.02 45 0.16 SD 0.18 0.14 0.11 0.15 0.13 0.14

N 24 24 24 22 22 22 1 M/Mdn 0.07 0.04 0.03 49 0.21 0.22 0.12 0.1 69 0.52 SD 0.19 0.13 0.15 0.22 0.15 0.19 N 28 28 28 24 24 24

2 M/Mdn 0.08 0.02 0.06 61 0.58 0.12 0.05 0.07 58 0.37 SD 0.11 0.12 0.1 0.21 0.03 0.19

N 19 19 19 12 12 12

3 M/Mdn 0.05 0.05 0 41 0.03 0.04 0.1 -

0.06 45 -0.38

SD 0.18 0.12 0.12 0.17 0.12 0.16

N 24 24 24 22 22 22

4 M/Mdn 0.06 0.03 0.03 52 0.30 0.09 0.08 0.01 49 0.08

SD 0.09 0.09 0.1 0.22 0.11 0.16

N 24 24 24 24 24 24

5 to 9 M/Mdn 0.03 0.02 0.01 52 0.13 0.08 0.06 0.02 64 0.13

SD 0.15 0.14 0.08 0.21 0.1 0.17

N 28 28 28 25 25 25

Total M/Mdn 0.06 0.03 0.03 51 0.27 0.1 0.07 0.03 54 0.17

SD 0.16 0.12 0.11 0.21 0.12 0.17

N 147 147 147 147 147 129 129 129 129 129

Source. NYCDOE Office of Accountability

* Achievement scores are converted to a proficiency-‐level scale that ranges from 1 -‐ 4.5 proficiency units. Diff. is the value-‐added modeling effect (VAM) calculated as mean difference between actual and predicted proficiency-‐level scores for similar students, in similar classes, and similar schools. **Median percentile is the percent of all NYC teachers with VAM effects lower than NYU graduates with the same number of total years of teaching experience.

***Bold font indicates the effect size meets the program criterion of 0.20, considered educationally meaningful but small. 5.0 DISCUSSION AND PLAN 5.1 Discussion


56

NYU’s five-year self-inquiry has been comprehensive and complex, as well as fruitful in terms of program impact, with more impact likely as the faculty continues to discuss the results of the inquiry. The inquiry used a variety of methods, measures, and perspectives to compile evidence that faculty have used and will continue to use in order to reflect on the efficacy of the NYU teacher education program. Evidence for the inquiry was collected in a series of individual research studies and periodic data collections over the five years, and presented to faculty both in Steinhardt and in Arts and Sciences in presentations, papers, information briefs, and reports, including the annual reports to TEAC. This information has been used formatively in program decision-making. We have also shared our methods and results in national fora such as CADREI and AACTE.

This Inquiry Brief is a summative assessment of NYU’s claims based on a cumulative compilation and analysis of the evidence. Across 78 analyses, the pre-determined program standards were met or exceeded for 54 (69.2%). Results were mixed for nine (11.5%). And standards were not met for 15 (19.2%). Mixed results are operationally defined as an even split of performance at/above standard and below standard for scales with multiple items. Therefore, overall standards were completely or partially met for nearly four-fifths of the measures/scales. Analysis and synthesis of these findings lead to the following conclusions about NYU’s claims:

Claim 1: Content Knowledge. The claim that NYU graduates are competent and qualified in content knowledge is strongly supported by the evidence for MA students and moderately supported for BS students. Claim 2: Pedagogical and Pedagogical Content Knowledge. The evidence strongly supports the validity of the claim of competence in pedagogical knowledge for both BS and MA students. Claim 3: Clinical Knowledge and Skill. The evidence pertaining to Claim 3 is more equivocal than the other claims, especially for BS students. Taken as a whole, however, the evidence provides moderate support for the claim of clinical competence. Claim 4: Caring Professional. The evidence provides strong support for the claim that NYU graduates—both BS and MA—are caring professionals. Cross-Cutting Themes. For both groups, the data support achievement of the standard for two of the three cross-cutting themes – life-long learning and multicultural attitudes/social justice – but not the third, use of technology for instruction.

5.2. Plan The findings of NYU’s 2011 TEAC Self-Inquiry and Internal Audit are voluminous. In most respects, they confirm the soundness and effectiveness of the University’s teacher education program, the strength of the quality control mechanisms it employs, and the validity of its claims. In many ways too they confirm the wisdom of new directions the faculty and administration have already set, beginning with the plan set forth in the first Inquiry Brief (IB), November 15, 2005. Progress on the implementation of the first IB plan is discussed below, followed by a description of the plan going forward.

5.2.1 Progress on the First IB Plan: NYU has made substantial progress implementing the plan from the first IB, including efforts to create more flexible and integrated information systems, to achieve greater transparency in the role that faculty communities of practice play in teacher education, to achieve greater integration of coursework and fieldwork and stronger ties


57

with schools and other programs where candidates intern, to create more opportunities for teacher candidates to study abroad, to forge even stronger links among the three departments that house the NYU teacher education program, to provide more frequent opportunities for raters to tune up their DRSTOS-R standards, and to put greater emphasis on central mechanisms that make NYU teacher education as a whole cohere. The following is a summary of progress on the eight specific components of the plan:

1. NYU has continued to upgrade the Steinhardt teacher education student database. Phoenix, the system in use when the first IB was written, has been replaced by Apprentice, which has more power and flexibility, and better integrates student information. Apprentice has greater capacity to track field placements and provides more accurate and timely information on student teaching assignments, including data on schools, field supervisors, and cooperating teachers.

2. NYU has expanded follow-study of its teacher education graduates. Follow-up study of graduates, which began with electronic matching to the New York State Education Department teacher data system, was enhanced by using systems that were more accurate and included more comprehensive data elements. These data were used in the evidence from the graduate tracking study reported above. In addition, follow-up surveys were conducted to obtain data on the employment and success of graduates who entered teaching and those who did not.

3. The effectiveness of graduates has been assessed through their impact on pupil learning. Cooperative agreements were established with the New York City Department of Education to obtain achievement test-score data for the pupils of NYU graduates who are teaching in grades 3-8 in the NYC public schools. These data have been used in value-added models of teacher effects, as reported in the results section above. NYU continues to explore the collection of pupil work as another measure of teacher effectiveness, an effort in which progress has been less advanced.

4. The assessment of student teachers with DRSTOS-R has been dramatically expanded. As of fall 2010, 63 field supervisors (70.8% of the total) had been trained to administer DRSTOS-R and submitted assessments for 371 (76%) of fall 2010 student teachers. Moreover, the assessment process has been integrated into the clinical training experience with data used to analyze student teacher progress during three-way conversations among field supervisors, cooperating teachers, and student teachers.

5. Communication and discussion of self-study results among faculty and field supervisors have increased. Steinhardt’s Center for Research on Teaching and Learning (CRTL) prepares comprehensive reports of DRSTOS-R data for the faculty of each certification area annually. The reports are discussed at program area faculty meetings and used to inform planning. Findings from other self-study measures are shared with faculty through written reports and presentations at faculty meetings and the Teacher Education Working Group.

6. CRTL is engaged in ongoing study of the linkages between specific components of Steinhardt’s teacher education program and the professional growth of its students. Exit and follow-up surveys have provided valuable insights into the courses and experiences that have been most and least useful in pre-service teacher preparation, as well as the students’ levels of preparation in specific essential teaching knowledge and skills. Some of this information has been reported in the survey findings presented in the results section above and has been used in faculty discussions aimed at continuous program improvement.


58

7. Steinhardt has increased the involvement of advanced graduate students in teacher education research. Three doctoral students in The Department of Teaching and Learning have been integrally involved in the research conducted at CRTL. They have participated in all aspects of the research and co-authored written reports and delivered conference presentations of the findings. The data from this research have been used in secondary analyses by two other doctoral students for independent research and three masters’ students for their theses.

8. NYU has expanded its partnerships with the New York City public schools for teacher education and research. NYU partnered with the City University of New York and the New York City public schools in a major initiative aimed at improving the alignment of pre-service teacher education with the needs of hard-to-staff public schools in the shortage areas of math, science, and special education. The partnership led to the design and redesign of core courses as well as content courses, the strengthening of the clinical field component, and collaborative co-teaching by university and public school faculty. In addition, the Research Alliance for New York City Schools was established at Steinhardt as a research consortium investigating priority educational issues using data from the massive and complex information systems maintained by the New York City Department of Education (NYCDOE). Also, CRTL forged agreements with the NYCDOE to obtain data to be used in the tracking of NYU graduates and the assessment of their value-added impact on their pupils’ test scores, as reported in the results section above.

5.2.2 The Plan Going Forward: The findings of this IB have raised—and will likely

continue to raise—many important questions that must be addressed to spur the program’s continued healthy development. These questions include, for example, the following:

• Are overall differences between BS and MA candidates’ performances a matter of maturity, admissions, programming, or some combination of these and other factors?

• Are program area differences in DRSTOS-R results a consequence of rater reliability faults, instrument shortcomings, curricular shortcomings, or some combination of these and other factors?

• How can we best increase our students’ sense of their own efficacy when it comes to such crucial dimensions of their prospective work as coping with the impact of poverty on their students’ lives, and of managing the complex dynamics of urban classrooms?

• How do we increase the amount, frequency, and quality of the clinical mentoring and coaching our candidates receive?

• What kinds of technology training and supports do our candidates need and how can we best provide them?

• How can we best ensure that our candidates are well prepared to work with parents? • How can we best ensure that our candidates are well prepared to work with English

language learners and students with disabilities? • How can we best prepare candidates to be knowledgeable about assessment practices and

data systems and to use data in instructional decision making? • How can we ensure that each of our candidates learns from an optimal balance of senior

scholars, highly effective practitioners, cutting-edge researchers, multiple field mentors, and highly skillful teaching coaches?


59

• What is the optimal balance and threshold of content knowledge and pedagogical content knowledge necessary for effective teaching at pre-K, K-6, and 6-12 teaching levels, and how do we ensure that all our candidates meet this standard?

• How do we ensure that all our teacher education programming is continually infused by good research on learning and teaching and that our graduates continue to seek out research that can inform their practice?

None of these questions is new for NYU, but the self-inquiry and internal audit set them before us with freshness and in some cases urgency. A recent back-page essay in Education Week, published by two of our colleagues, will inspire our deliberation (Alter & Pradl, 2011). Our methods and timelines for addressing these and other questions include the following:

• Reading of the Brief and appendices. The whole teacher education faculty as well as staff associated with the teacher education program, other members of the Teacher Education Council and Working Group, and the Deans group will read through and digest the voluminous findings of the self-inquiry and self-audit. Timeline: Summer and early fall 2011.

• Discussion and prioritization of issues and action steps. The faculty will discuss issues and action steps within and across faculty communities of practice, using a summary of the Brief and key appendices prepared by the TEAC Coordinators. Timeline: September & October 2011.

• Implementation of the most immediate action steps. Are there decisions that the department curriculum committees, the Deans, the TEWG, the Teacher Education Council, and so on, can make to address needs or problems identified in the findings of the self-inquiry and internal audit? Timeline: November & December 2011.

• Preparation for and response to the external TEAC audit. What short-term improvements to the QCM would make for a more efficient and productive audit? What immediate and long-term action steps do findings from this audit suggest? Timeline: September 2011 –February 2012.

• Long-term strategic planning based on findings from the self-inquiry and internal and external audits. Timeline: January to May 2012.

• Examination of the results of an internal evaluation of our teacher education programs using standards and indicators of NCTQ.


60

6.0 REFERENCES

Adelman, C. (1995). The new college course map and transcript files: changes in course-taking and

achievement, 1972-1993. Based on the post-secondary records from two national longitudinal studies. Washington, D.C.: Department of Education.

Adelman, C. (2001). Putting on the glitz: How tales from a few elite institutions form America’s

impressions about higher education. Connection, 15(3), 24. Alter, M. & Pradl, G. M. (2011). Where’s the red queen: Ending three-card Monte in teacher

education. Education Week, May 18. Brabeck, M. (2008). Why We Need ‘Translational’ Research: Putting Clinical Findings to Work in

Classrooms. Education Week, May 21. Brabeck, M.M. & Shirley, D. (2003, January). Excellence in education schools: An oxymoron? Phi

Delta Kappan, 368-372. Burant, T.J., Chubbuck, S.M., & Whipp, J.L. (2007). Reclaiming the moral in the dispositions debate.

Journal of Teacher Education, 58(5), 397-411. Cohen, J. (1988) Statistical power analysis for the behavioral sciences. (2nd edition). Hillsdale, NJ:

Lawrence Erlbaum Associates. CTB/McGraw-Hill (2009). New York State testing program 2009. Technical reports. Retrieved on

May 30, 2011, from http://www.p12.nysed.gov/apda/reports/2009/ela-techrep-09.pdf And also from http://www.p12.nysed.gov/apda/reports/2009/math-techrep-09.pdf Danielson, C. (1996). Enhancing professional practice: A framework for teaching. Alexandria, VA:

ASCD. Danielson, C. (2007). Enhancing professional practice: A framework for teaching, 2nd ed.

Alexandria, VA: ASCD. Davis, B., Sumara, D., & Luce-Kapler (2000). Engaging minds: Learning and teaching in a complex

world. Mahwah, NJ: Lawrence Erlbaum. Gibson, S., & Dembo, M. H. (1984). Teacher efficacy: A construct validation. Journal of Educational Psychology, 76(4), 569-582. Grossman, P. and Loeb, S. (Eds.) (2008), Alternative routes to teaching: Mapping the new landscape

of teacher education. Cambridge, MA: Harvard Education Press.


61

Hummel-Rossi, B., Tobias, R., & Ashdown, J. (2009). Creating usable evidence to improve teacher education programs serving urban public schools. A paper delivered at the annual meeting of the AERA, San Diego, April. CRTL Research Report Series RR-0409-1.

Hummel-Rossi, B. Tobias, R., Ashdown, J., & Smith, A. (2008). Teacher education’s responsibility to

its metropolitan constituents: A longitudinal value-added study. A paper presented at the annual meeting of AERA, New York City, March 2008. CRTL Research Report Series RR-0308-1.

Jeffery, J. & Tobias, R. (2009). Circle of inquiry: partnership researchers’ perspectives on school-

university collaborative processes. (2009). A paper delivered at the annual meeting of the AERA, San Diego, CA, April 2009.

Kane. T. J., Cantrell, S., et al. (2011). Initial findings from the measures of effective teaching project.

Seattle, WA: Bill & Melinda Gates Foundation [http://www.metproject.org/downloads/Preliminary_Findings-Research_Paper.pdf]

Levine, A. (2006). Educating school teachers. Washington, D.C.: The Evaluation Schools Project. Levine, A. & Cureton, J. (1998). When hope and fear collide: A portrait of today’s college student. San Francisco, CA: Jossey Bass. McDonald, J. P., Tobias R., Pietanza R, Jeffery J. & Soponis, T., (2011). Principles of partnership:

How theory and practice can mix well in teacher education. Manuscript in preparation. Meier, J. & Crowe, E. (2009) Evaluation of partnership for teacher excellence. An evaluation report

prepared by Arete Consulting for the Petrie Foundation. New York City Council (July 2009). A staff report of the New York City Council Investigation

Division on teacher attrition and retention. Retrieved on May 30, 2011, from http://www.nyc.gov/html/records/pdf/govpub/1024teachersal.pdf

National Council for Accreditation of Teacher Education (NCATE) (November 2010). Transforming

teacher education through clinical practice: A national strategy to prepare effective teachers. Washington, D. C.: NCATE.

New York State United Teachers (2001). Teacher certification in New York State. Information

Bulletin no. 200112. Retrieved June 1, 2004 from http://nysut.org/research/bulletins/teacher cerrification.html#introduction.

Onwuegbuzie, A.J., et.al (2007). Students’ Perceptions of the characteristics of effective college

teachers: a validity study of a teaching evaluation form using a mixed-methods analysis. American Education Research Journal, (44)1, 113-160.

Poliakoff, A. R., Dailey, C. R., & White, R. (2009). Cross-case study of Teachers for a New Era

Learning Network universities. Washington, D.C.: Academy for Educational Development.


62

Polleck, J. and Jeffery, J. (2010). Reciprocity through co-instructed site-based courses: Perceived challenge and benefit overlap in school-university partnerships. Teacher Education Quarterly, 37(3), 81-99.

Ponterotto, J.G., Baluch, S., Greig, T., and Rivera, L. (1998) . Development and initial score validation of the teacher multicultural attitude survey. Educational and Psychological Measurement, 58(6), 1002-1016.

Rogers-Sirin, L. & Sirin, S. R. (2009). Cultural competence as an ethical requirement: Introducing a new educational model. Journal of Diversity in Higher Education, 2(1), 19-29.

Schon, D.A. (1983). The reflective practitioner: How professionals think in action. New York: Basic.

Schon, D.A. (1987). Educating the reflective practitioner. San Francisco: Jossey Bass.

Sirin, S. & Collins, B. (2009). Graduate teacher education at New York University: An exploration of backgrounds, perceptions, and expectations. New York University: New York, 2009.

Steele, J. L., Hamilton, L. S. & Stecher, B. M (2010). Incorporating student performance measures into teacher evaluation systems. Santa Monica, CA.: The Rand Corporation. Suarez-Orozco, C. & Suarez-Orozco, M. M. (2002). Children of immigration. Cambridge, MA:

Harvard University Press.

Taub, A., Tobias, R. & Mayher, J. (2005). Inquiry brief: a self-study of NYU’s teacher education programs. A report to TEAC for initial accreditation, November.

Tian, J. & Tobias, R. (2009). Student evaluations of co-taught, off-site inquiries courses: fall 2009.

CRTL Program Evaluation Series report PE-2010-01. Tobias, R. & Bang, H. First-year feedback survey of spring 2007 NYU teacher education graduates.

CRTL Research Report Series RR-0608-1, June 2008. Tobias, R., Hummel-Rossi, B., Ashdown, J., Simic, O. & Woo, K. (2010). Accountability in teacher

education: ecological analyses of VAM effects. A paper presented at the annual meeting of the AERA, Denver, CO, May 2010. CRTL Research Report Series RR-0410-01, April 2010.

Tobias, R. & Miller, R. (1999). The relationship between accountability, measurement

scale, and grade inflation in school quality review ratings. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Quebec, Canada, April 19-23.

Tobias, R. & Tian, J. (2010). Program exit survey of May 2010 NYU teacher education graduates. CRTL Program Evaluation Series report PE-1110-01, November 2010.


63

Tobias, R. & Tian, J. (2009). Graduating students’ perceptions of the effectiveness of their teacher education programs: Class of 2009. CRTL Program Evaluation Series report PE-0909-01, September 2009.

Tobias, R., Tian, J. & Saad, B. (2010) One-year follow survey of Steinhardt teacher education

graduates: Class of 2009. CRTL Program Evaluation Series report PE-0410-01, April 2010. Tobias, R. & Tian, J. & Woo, K. (2009). First-year feedback survey of spring 2008 NYU teacher

education graduates. CRTL Research Report Series RR-0909-1, September 2009.

Tobias, R., Woo, K. & Pignatosi, F. Are we developing high quality teachers and can we prove it? A study of the validity and utility of a pre-service teacher assessment system. A paper presented at the annual meeting of the AERA, San Diego, April 2009. CRTL Research Report Series RR-0409-2.

Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct.

Teaching and Teacher Education, 17, 783-805. Villegas, A.M. (2007). Dispositions in teacher education: A look at social justice. Journal of Teacher

Education, 58(5),370-380. Wyckoff, J., Boyd, D., Grossman, P., Hammerness, K., Lankford, H., Loeb, S., McDonald, M.,

Reininger, M. & Ronfeldt, M. (2008). Surveying the landscape of teacher education in New York City: Constrained variation and the challenge of innovation. Educational Evaluation and Policy Analysis, 30, pp. 319-‐343.

Wyckoff, J. , Boyd, D., Grossman, P., Lankford, H. & Loeb, S. (2009). Teacher

preparation and student achievement. Educational Evaluation and Policy Analysis, 31, pp. 416-440.


64

Appendix A

Internal Audit INTRODUCTION

On October 5, 2010, the TEAC coordinators (Professors Tobias, McDonald, and Pietanza) met with Mary Brabeck, Dean of the Steinhardt School of Culture, Education and Human Development, and with Heather Herrera (Assistant Director for Curriculum Development) to review the TEAC guidelines and timeline for completing the Inquiry Brief. At the meeting, Dean Brabeck suggested that we invite Alyson Taub, Professor Emerita of NYU and former NYU TEAC coordinator, to serve on the internal audit team. The TEAC coordinators discussed the composition of the internal audit team at the November 30, 2010 meeting of the NYU Teacher Education Working Group (TEWG). TEWG approved the selection of Taub, and also the following members of the Steinhardt faculty/staff (Anne Ballantyne, David Montgomery, Anne Burgunder, and Mary McShane). TEAC Coordinator Rosa Pietanza also agreed to serve as an auditor, as well as Coordinator of the Internal Audit. At this meeting, TEWG also approved the procedures for the audit and the description of the quality control system presented below. Finally, members discussed and approved details of a Shadow Protocol. Not part of the Internal Audit itself, this experience was designed to use the occasion of the Audit as a parallel opportunity for faculty learning and inquiry (Quality Principle II).10 The Shadow Protocol culminated on March 23 in conversations by the entire teacher education faculty meeting in communities of practice (COPs). These conversations centered on low-‐inference accounts by trained doctoral students of the experience over one or two days of one of the COP’s students whose transcript also figured in the Internal Audit. Twelve students, or one third of the audit sample chosen at random, were shadowed.

On January 31, 2011, an orientation was held for the Internal Audit team. At this meeting, the TEAC Coordinators (Tobias, McDonald, and Pietanza) discussed the principles and goals of TEAC, and presented an Internal Auditor’s notebook providing background materials. They also reviewed the Quality Control System, the planned probes, and the audit trail. The Internal Audit itself took place on February 1 and 2, 2011, led by Professor Pietanza with assistance from Tobias and McDonald, Judith Costello (Director of Institutional Research), and many other members of the NYU faculty and staff as detailed in the findings. On the afternoon of February 2, the auditors prepared an initial set of findings. Then, over the next month, working through email exchanges and phone calls, they elaborated on the findings and prepared a set of questions to guide faculty discussion of them. The final set of findings and questions was available on March 7. These informed a meeting of the entire NYU teacher education faculty on March 23. This was also the meeting at which the Shadow Protocol conversations took place. At the same time, the entire set of findings and questions were sent to members of the Teacher Education Working Group (TEWG) in advance of its meeting on March 28. Under the plan for the Internal Audit, TEWG – a group of teacher education faculty leaders and key staff – took principal responsibility for assembling answers to the Auditors’ questions. Professor McDonald, as Chair of TEWG, took responsibility for ensuring that the answers to the Auditors’ questions were accurate and fully vetted by the most relevant (per topic) members of the administration, staff, and faculty.

By April 29, TEWG had completed a draft of Appendix A with most of the answers supplied. This draft

10 The Shadow Protocol was based on a professional development model that NYU faculty use in partner schools (McDonald, Mohr, Dichter & McDonald, 2007). Shadowers produce low-‐inference accounts of a “day in the life” of the shadowees based on observations as well as a running interview and work sampling, and present these accounts to stakeholders (in this case, the COP overseeing the shadowee’s teacher education program area) as input for discussion of some question of interest. In this case, the question of interest was, “How does this teacher education candidate perceive the integration of coursework, fieldwork, and advisement?”


65

was then posted on the NYU TEAC Blackboard site for review by all members of the teacher education faculty, of the Steinhardt administration, and of the University-‐wide Teacher Education Council. Changes and additions were suggested by various faculty, staff, and deans over the next week. These centered for the most part on clarification, and in some cases on errors. Appropriate changes and additions were then incorporated into a penultimate draft of Appendix A, which was posted on the TEAC Blackboard site on May 4. All members of the teacher education faculty were then asked to provide final feedback on this draft in advance of the preparation of a final draft. With that final draft in place, the faculty voted electronically on the Brief and all its appendices during the week of May 23, 2011. AUDIT PROCEDURES

To assess the Quality Control System (QCS) as it impacts undergraduate and graduate students, the Internal Audit focused on the experience of 36 students currently enrolled in teacher education. In selecting this audit sample, Robert Tobias, Director of the Center for Research on Teaching & Learning, followed a sampling procedure within the following population constraints: junior and senior undergraduates, and graduate students in their second or third semester of study. These constraints were intended to capture as much contact as possible with the diverse systems of the program, i.e., methods courses, content courses, fieldwork, and advisement. Tobias first created a sampling frame by using SPSS Report for the 438 students in the research population, stratifying by degree and program. He then used a table of random numbers to select the specific students for the audit sample (Stattrek.com:/Tables/Random.aspx). More specifics on the procedure for selecting the audit sample are available in the TEAC Auditor’s Files and in the CRTL.

The Internal Audit was designed to test if the QCS elements outlined below operate effectively and efficiently. Each transcript was treated as a record of interaction with these elements. For example, the student took a course in the methods of teaching social studies. Was the course approved by the faculty? Was the instructor duly appointed? Does the course conform to program requirements? Did the course meet in an instructionally appropriate environment? And so on. A set of 28 probes, covering the 4 domains of the QCS (affecting curriculum, faculty, candidates, and resources) was constructed by the TEAC Coordinators, drawing in the process on input from faculty. Each of the 6 auditors was randomly assigned 6 transcripts, printed from the Student Information System. Using a random number generator, the auditors then selected 4 probes for each of their assigned transcripts (one per each of the 4 QCS domains). The result was a total of 144 tests of the QCS.

Because some probes were specifically directed to undergraduate or graduate transcripts, second drawings were sometimes needed. The 28 probes were -‐ with one exception -‐ used multiple times (from 2 to 12 times per probe with a median usage of 4). And by the Auditors’ account, all key elements of the QCS were tested multiple times. Overall, the Internal Audit proved to be a significant test of the QCS.

The Internal Audit was conducted over two days (February 1 and 2, 2011). Conferring with each other as well as others as needed, the auditors pursued their assigned probes, following as parsimonious a route as possible through the systems of NYU teacher education. As needed, they met with deans, department chairs, program directors, student advisors, faculty members, admissions staff, administrators, Center for Research on Teaching and Learning (CRTL) staff, Office of Clinical Studies staff, the NYU Certification Officer, and others. In addition, they examined documents and webpages. In some cases, probes required more information gathering and checking than could be managed within the two days of the audit itself. Thus the auditors continued on the audit trail via email, with the assistance of the Audit Coordinator, for approximately two weeks following the audit – until all probes drawn per student had been concluded to the auditors’ satisfaction.


66

In following each probe, the auditors sought verifiable evidence. For example, if a staff member consulted as part of the audit mentioned a particular document, the auditor would ask to see the document. Auditors recorded the probes they selected, the mechanisms they probed, and their findings on the Audit Trail-‐Quality Control Mechanism Charts (one form completed per student in the sample). The Charts are available in the TEAC Coordinators’ office. The probes and findings are listed on pages 72-‐91.

During the afternoon of February 2, the auditors discussed with each other all findings (to date) per probe, and in the process verified that they had collectively and satisfactorily tested each element of the QCS. They also formulated an initial set of questions (later expanded by email), as well as a set of summative findings and some recommendations for action steps. It is important to note that the latter were thus formulated before the questions were fully developed or answered. However, the auditors felt there would be value for the faculty in hearing their immediate impressions based on audit findings alone, and indeed these recommendations proved to be helpful input to the process of answering the questions.

That process of answering questions constitutes for the purpose of this Brief the faculty’s Discussion of the Internal Audit results. As the answers themselves (listed in Table A.1) may suggest, however, the discussion that began with the effort to answer these answers has continued into the 2011-‐2012 academic year and is likely to continue beyond it. The Internal Audit proved, in other words, to be a strong stimulus to ongoing faculty inquiry and learning. The cycle of the Internal Audit as described above, and as imagined on a biannual basis (following one of the auditors’ recommendations) is captured below in Figure A.1.

OVERVIEW OF THE QUALITY CONTROL SYSTEM BY DOMAIN

The elements that comprise the NYU Teacher Education Quality Control System (QCS) are presented in the following overview in four domains: curriculum, faculty, candidates for teacher certification, and resources.

Complete IA Add to findings Raise queslons Issue findings & queslons to

TEWG

Learn from IA Answer queslons Consider aclon

steps Take aclon

Conduct IA Draw sample Create probes Draw & pursue

probes List findings


67

Curriculum Faculty communities of practice (COPs), led by Program Area Directors (e.g. Art, Childhood, English Education, etc.) oversee curriculum quality at the program-‐area level. They update the bulletin and course planning sheets, review course evaluations in their area, review syllabi, manage scheduling and teaching assignments of COP members, recruit adjunct instructors as needed, work closely with the Office of Clinical Studies to ensure the quality of field placements in their area, and work closely with the Office of Academic Affairs to ensure compliance with state policies. http://steinhardt.nyu.edu/programs/#search:teacher_certification Each of the COPs is nested within one of three departments which collaborate in the management of teacher education at NYU. The Chairs of the departments of Teaching & Learning, Art & Arts Professions, Music& Performing Arts Professions appoint and oversee program area directors, and their department Administrators ensure department-‐level quality control processes (for example, course design, registration procedures, syllabi collection, and curriculum committees). http://steinhardt.nyu.edu/art/ http://steinhardt.nyu.edu/music/ http://steinhardt.nyu.edu/teachlearn/ Steinhardt requires that all courses be approved by department-‐level curriculum committees and that such committees be represented on a school-‐wide Committee on Courses & Programs. The latter reviews course proposals against school-‐wide criteria only, such as limiting course redundancy, increasing cross-‐departmental collaboration, and addressing school-‐wide needs. http://steinhardt.nyu.edu/governance/bylaws http://www.nyu.edu/registrar/listings/. Policies and procedures related to quality in the field components of the teacher education program at NYU are overseen by the Office of Clinical Studies, a school-‐wide office that cultivates, assigns, and evaluates all clinical placements (pre-‐student teaching or Learning Partners placements, and student teaching placements); appoints and supports supervisors of placements; and directs the NYU Partnership Schools Program. http://steinhardt.nyu.edu/teacher_education/ http://steinhardt.nyu.edu/petrie/partnershipschools The quality of the content base of teacher education at NYU is overseen by the faculty communities of practice as well as the Teacher Education Council, co-‐chaired by the Deans of the Steinhardt School and the College of Arts and Science. The Council is comprised of an equal number of faculty from Steinhardt and Arts & Science. The Steinhardt faculty are members of the three departments that collectively manage the teacher education program (Teaching & Learning, Art & Arts Professions, and Music & Performing Arts Professions), and of the three allied departments that support the program (Humanities & Social Sciences in the Professions, Applied Psychology, and Administration, Leadership & Technology). The Arts & Science faculty come from the College of Arts & Science, the Polytechnic Institute of NYU, and the Courant Institute for Mathematical Studies. The Council advises the Deans and faculty on all matters related to teacher education, including program and curriculum development and evaluation. It meets twice yearly. www.Steinhardt.nyu.edu/secure/teac [public web page under construction as of Sept. 29, 2011] The quality of the core curriculum of teacher education at NYU – including field studies and cross program area requirements – is monitored by the Teacher Education Working Group (TEWG), a standing committee of the Teacher Education Council. TEWG meets at least twice each semester.


68

www.Steinhardt.nyu.edu/secure/teac [public web page under construction as of Sept. 29, 2011] The Undergraduate Curriculum Advisory Committee reviews new undergraduate degree programs with respect to NYSED policies, the Graduate Commission reviews new graduate degree programs. http://www.nyu.edu/provost/about.office/committee.ucac.html http://gsas.nyu.edu/object/grad.about.apr http://steinhardt.nyu.edu/teachlearn/certification/ http://steinhardt.nyu.edu/programs/#search:teacher_certification The management of the NYSED connection at NYU regarding teacher education involves the office of the Assistant Director for Curriculum who works closely with the faculty, the school-‐wide Committee on Courses and Programs, and the Assistant Provost for Academic Program Review, to ensure that all courses and programs, including teacher education, comply with school, university, and state requirements. http://steinhardt.nyu.edu/academics/affairs/course_proposal/ Internal evaluation efforts – including those associated with course evaluation and accreditation – are conducted by the Center for Research on Teaching and Learning (CRTL), a school-‐wide research center. Student teachers complete ETFQ (End of Term Feedback Questionnaire) that provides feedback on supervisors and cooperating teachers. The feedback is reviewed the Director of the Office of Clinical Studies and is shared with Program Area Directors. CRTL also provides student exit data, and follow-‐up graduate data to faculty, as well as performance data on State Certification Exams. http://steinhardt.nyu.edu/teachlearn/crtl Faculty

Working with Program Area Directors, the Chairs of the three Steinhardt departments with teacher education program areas manage hiring, promotion, and tenure matters at the department level as well as peer observation and review procedures. http://steinhardt.nyu.edu/art/ http://steinhardt.nyu.edu/music/ http://steinhardt.nyu.edu/teachlearn/ http://steinhardt.nyu.edu/faculty_affairs/tenure_evaluation The teacher education faculty hiring and evaluation procedures are administered by the Office of Faculty Affairs and the Office of the Provost. http://steinhardt.nyu.edu/faculty_affairs/ptguidelines http://www.nyu.edu/content/dam/nyu/compliance/documents/FacHbk2008.pdf http://steinhardt.nyu.edu/faculty_affairs/professional_development_fund_request http://steinhardt.nyu.edu/faculty_affairs/for_new_faculty http://steinhardt.nyu.edu/faculty_affairs/adjunct_faculty http://steinhardt.nyu.edu/adminfinance/howdoi-‐2010-‐02-‐22-‐IDA http://www.nyu.edu/about/leadership-‐university-‐administration/office-‐of-‐the-‐president/office-‐of-‐the-‐provost.html Appointments of student teaching supervisors and cooperating teachers, and evaluation of their effectiveness, is overseen by the Office of Clinical Studies http://steinhardt.nyu.edu/scmsAdmin/uploads/001/373/field_handbook.pdf


69

Candidates Admission to the NYU teacher education program at the graduate level is managed by the Steinhardt Office of Graduate Admissions which recruits applicants in partnership with the COPs. Admission guidelines include a transcript review verifying at least 30 credits of content coursework required for initial certification by New York State, and an overall GPA of at least 3.0. http://steinhardt.nyu.edu/graduate_admissions/ http://steinhardt.nyu.edu/portal/future_undergraduates http://www.nyu.edu/admissions.html Program Area Directors are the key agents, typically in collaboration with the Office of Clinical Studies, Student Advisors, and the Office of Student Affairs, in recommending that candidates be terminated from the program. http://steinhardt.nyu.edu/programs/#search:teacher_certification Student advisement for NYU teacher education is organized by department, but connected to school and university resources through the Office of Student Affairs, and a school-‐wide advisement community of practice that meets periodically. Advisement systems vary by department and level (BS or MA). In all three departments, advisors meet with advisees on a compulsory basis prior to registration and clear them to register for courses. http://steinhardt.nyu.edu/advisement/ NYU’s university wide Student Information System (SIS) is currently in the process of being replaced by a Peoplesoft system, with migration to the new system continuing through the 2011-‐2012 academic year. Multiple changes include new course numbers university wide, greater access of faculty to student information, online grade submission, and many others. http://www.nyu.edu/registrar/sis/docs/SIS_NextGen_Project_Overview.pdf Registration services, counseling services, career services (including teacher certification services), academic support services, and social services for teacher education candidates are managed by the Steinhardt Office of Student Affairs in collaboration with university-‐wide offices (especially health services, career placement services, and services for students with disabilities. http://steinhardt.nyu.edu/studentaffairs/dean http://www.nyu.edu/shc/ http://steinhardt.nyu.edu/counseling/career http://steinhardt.nyu.edu/teachlearn/job_strategies http://www.nyu.edu/life/safety-‐health-‐andwellness/students-‐with-‐disabilities.html The Teacher Certification Officer in the Office of Student Affairs advises in matters of certification, and officially recommends program completers for initial certification in New York State. http://steinhardt.nyu.edu/teachlearn/certification/ http://steinhardt.nyu.edu/programs/#search:teacher_certification The integrity of academic policies and the student information system are maintained and monitored by the Office of the University Registrar. The computerized advising/degree progress support system, "On Course", evaluates course work taken against degree requirements to determine progress toward the completion of a degree. The Director of Registration Services acts as the liaison between NYU Steinhardt and the University Registrar and Bursar in matters regarding registration, licensing, grading and graduation.


70

His is the final signature on registration forms and change of grade forms. http://steinhardt.nyu.edu/registration/ http://www.nyu.edu/registrar/transcripts-‐certification/degree-‐progress.html. http://www.nyu.edu/registrar/about/ http://www.nyu.edu/registrar/graduation/apply.html. Resources

Each Steinhardt department, including the three that host teacher education program areas, has a chief administrator as well as other administrators who maintain the crucial interface between the larger resource systems of the School and the University, on the one hand, and the department faculty. For example, they ensure that full-‐time faculty have well equipped offices and that part-‐time faculty have plentiful access to work space and work tools. They manage the many human resource details associated with part-‐time faculty, and with the department chairs, they plan and administer budgets. http://steinhardt.nyu.edu/art/ http://steinhardt.nyu.edu/music/ http://steinhardt.nyu.edu/teachlearn/ The management of resources for teacher education at a school-‐wide level is with the Steinhardt Office of Administration and Finance. The purview of the Office includes non-‐faculty human resource management, non-‐classroom space management, financial planning and operations, institutional research, and facilities and technology support services for the three departments housing teacher education program areas. http://steinhardt.nyu.edu/adminfinance/ Classroom assignment operations are centralized at NYU. Room assignments are made based on the number of students registered in the course, and are monitored for compliance with room occupancy codes. Steinhardt Registration Services troubleshoots as needed. http://steinhardt.nyu.edu/registration/


71

FIGURE A.2 INTERNAL AUDIT INTERACTION WITH QCS ELEMENTS

Office of Clinical Studies/ Apprentice

Selection, Support, Mentoring

Promotion

Transcripts

Lead to

Dept. Chairs

Communities of Practice

CRTL

Curriculum

Tenure & Tenure Track

Non Tenure

Track

CRTL

Director of Faculty Affairs

Affairs

Annual Evaluations

Program and Career Advisement

Student Support Services

Faculty and Staff Advisors Resources Candidates

Begin with random selection of BS and MA Education

Students SStudents

Field Experience

Field Placements& Supervisors

Facilities, Classrooms Equipment

Tech., Library, Media

Library

Faculty, Funding

Course Syllabi, Web pgs., Program Descriptions

Curriculum Committees, Office of Acad. Affairs

Adjunct

Office of Clinical Studies

Certification Office

Faculty

Undergrad and Grad Admissions

Program Area Directors

Requirements


72

AUDIT FINDINGS The comprehensive compilation of probes, quality control mechanisms probed, and findings are listed below. They are followed by the

auditors’ summative findings composed in the week following the audit, and the recommendations they drew up on the last afternoon of the audit. Table A.1 provides the answers to the auditors’ questions which the Teacher Education Working Group (TEWG) generated, in consultation with other faculty and staff, beginning at its meeting on March 28, 2011, and concluding on April 29, 2011. Thus the logic of the NYU Internal Audit overall was that the fresh perspectives of the auditors, captured in their specific findings, summative findings, questions, and recommendations, would serve as spurs to the faculty’s reflective answers to the questions, and to their agenda setting for further faculty learning and inquiry, as well as program improvement efforts. Findings by Internal Auditors in Response to 28 Probes11

Curriculum Probe 1. Verify the program area in which the student is matriculated, and look for evidence of a faculty community of practice that meets regularly in order to ensure that the program is effective and aligned with research and best practice. QCS elements probed:

• Faculty communities of practice (COPs) • Program Area Directors

Findings: Two program areas were probed at random. One auditor interviewed the co-‐director for the Math Ed. Faculty and was informed that the COP meets twice a year, agendas are provided and minutes of meetings are available and math supervisors meet every month. The auditor searched for the math COP information on the NYU website, and noted that the composition of the COP is not listed. A second auditor interviewed a member of the Ed. Theatre faculty, who confirmed that the Ed. Theatre COP meets twice a semester and is situated within a larger ARTS Education faculty (music, dance, theatre, fine arts) and has contact with other faculty groups – including Teaching & Learning, Public Health, and Gallatin. Probe 2. Choose the fourth course listed on the transcript, and obtain a copy of the syllabus. Check the syllabus against the original course design approved by the faculty. Discuss any discrepancies with the program area director or instructor.

11 Note that in answering the auditors’ questions, the faculty and other members of TEWG as well as the Teacher Education Council found a small number of errors in the findings – though not ones that seemed consequential in terms of the questions the auditors raised. Except in one or two cases where a small error might prove confusing, the Coordinators have let the original findings stand, though they have sometimes flagged errors in their answers to the questions (Table A.1).


73

QCS elements probed:

• Faculty communities of practice (COPs) • Office of Clinical Studies • Assistant Director for Curriculum Development (Office of Academic & Faculty Affairs)

Findings: Four courses were probed. For E85.0006, Aural Comprehension in Music I, the course description was available. For E85.2092, Field Observation, an auditor learned that the course was recently developed to support pre-‐student teaching fieldwork. A short syllabus statement was provided by the instructor of record. The course does not appear in the Steinhardt course description database. The course number appears on the Music Ed website. [TEAC Coordinators later found that the course does appear in the Steinhardt course description database maintained by the Assistant Director for Curriculum Development. However, the instructor had provided the auditor with the wrong course number.] For E27.2999, Social Responsibilities for Educators, the course description was on file with the Assistant Director for Curriculum Development. The course description on the syllabus ties closely to course description on file. For G45.9891-‐001: Independent Guided Reading, Steinhardt staff do not have access to information concerning A&S coursework, so the auditor was not able to obtain data from SIS as to the instructor, or otherwise complete the probe. And the Steinhardt Assistant Director of Curriculum had no information concerning the course design. Probe 3. Find the first field placement listed on the transcript – whether pre-‐student teaching or student teaching. Determine how the placement was made, and what evidence exists regarding the effectiveness of the placement. QCS Elements probed:

• Faculty communities of practice (COPs) • Staff and faculty advisors • School-‐wide Committee on Courses and Programs • Office of Clinical Studies • Apprentice system database that tracks field work • Center for Research on Teaching and Learning (CRTL)

Findings: Four auditors drew this probe and divided tasks. One auditor interviewed a staff member in the Office of Clinical Studies who traced the placement process for pre-‐student teaching and student teaching via Apprentice-‐a software system that tracks individual students’ field placement history. The students probed had registered in Apprentice to request field placements. They listed specific schools or school-‐ type preferences. Staff and in some cases faculty considered the student preference in the context of program requirements and available


74

placements. Three evaluation tools were used to assess effectiveness of the placement: DRSTOS-‐R (a summative evaluation of the student teacher’s skills), ETFQ (End-‐of-‐term-‐feedback questionnaire, or student teacher’s evaluation of the placement), and an evaluation by the supervisor of the placement. Students were asked to complete evaluations of cooperating teachers (CTs) and Supervisors by logging into Apprentice and following link to Survey Monkey. Another auditor tracked DRSTOS evaluation for 2 students. The Director of the CRTL provided the first student’s DRSTOS evaluation, as completed by the student’s Math student teaching supervisor, and the student teacher’s evaluation of the placement. Both documents indicated a successful placement. The Director also provided the second student’s DRSTOS evaluation (which indicated success in the Music Candidate’s student teaching placement), but could not provide the student’s evaluation of the placement (likely because the student failed to submit the online evaluation). A third auditor interviewed a field placement coordinator and learned that at the time of the last TEAC accreditation audit, the Office of Clinical Studies operated entirely within the Department of Teaching & Learning. One result of that audit and follow-‐up work by the Teacher Education Working Group (TEWG) was the addition of a staff member to the Office to handle arts placements. Today that staff member, working with faculty, handles placements in art, dance, and educational theatre, but stays in touch with music only through the Apprentice system and occasionally by suggesting schools. The auditor spoke with a faculty member in music who explained why music is not currently involved. This faculty member makes all the placements himself – 23 this semester. He explained that he knows the group of cooperating teachers and their strengths, that he talks to all the students personally to determine good matches, and then maintains contact with both the students and the cooperating teachers throughout the placement. Auditors also requested End of Term Feedback Questionnaire (ETFQ) results on 6 students. ETFQs were found for four of the students and in all 4 cases the CTs were rated as being excellent. The supervisors were rated: 3 good and 1 excellent. Two ETFQs were not located (likely because the students had not submitted them).

Probe 4. Choose the first general pedagogical core course or program requirement listed on the transcript, and look for evidence of a faculty community of practice that meets regularly in order to ensure that this course is effective and aligned with research and best practice.


75

QCS elements probed: • Program Area Directors • COPs

Findings: Auditors probed four pedagogical core courses and program requirements, and interviewed program directors as follows: The first pedagogical core course probed was Foundations of Educational Linguistics, a course staffed by the English Education faculty. The Program Director reported that three faculty COP meetings had been held during the fall semester which is typical, and provided minutes. Adjunct professors attended one of these meetings. In addition, most adjunct professors, all supervisors, and two tenure-‐track faculty members attend monthly supervisor meetings. The course selected was taught by an adjunct professor who is also an advanced doctoral student actively involved with all members of the faculty COP. The second core course was Inquiries into Teaching & Learning 3. According to the course Co-‐Director, all current co-‐instructors are either full-‐time faculty or adjunct faculty; half are also currently practicing NYCDOE teachers. As a COP, they meet three times a semester to discuss issues and work on the continuing development of the course. The co-‐directors also visit each section once a semester to assess the nature of the teaching environment and address any needs the teaching team may have. Course evaluations are also reviewed by the Co-‐Directors. The third course was E26.2001, Language & Literacy – a core course for all secondary students outside English Education (English students take multiple courses in literacy). According to the Director of English Education, this course was transferred mid-‐fall semester 2010 from the Literacy program whose faculty COP is focused on childhood and early childhood literacy to the English Education COP. It will undergo evaluation and likely transition beginning this spring. Because it attends to language development and literacy, it will likely come under the control of a joint faculty group, using a COP model like the one that has historically overseen the Inquiries course. The fourth course probed was Curriculum Trends in High School and College Mathematics. The Math Program Area Director reported that she and the faculty COP are looking at all the math courses, the sequencing of courses, the requirements, etc., with an eye to developing a more coherent overall pattern. The particular course involved here had major changes to it recently. The Program Area Director said that she taught it but admitted that it wasn’t fitting the needs of the students. End-‐of-‐term course evaluations and also feedback from students said that it involved too much of a content focus. After discussions, the faculty decided that the course needed to focus more on teaching math, i.e. how to observe, plan, etc. with the particular content in mind. Talking to another full-‐time math faculty member, an auditor was informed that the three faculty members attend Supervisor Meetings every semester (though the Program Director does not always attend), and program-‐wide COP meetings are “occasional.”

Probe 5. Choose the eighth course listed on the transcript and obtain a copy of the course evaluation for that semester. QCS elements probed:

• CRTL


76

Findings: Four courses were probed by auditors and they met with staff and the Director of CRTL. The eighth course on a BS Childhood/Special Ed. transcript was Inquiries into Teaching and Learning. All evaluations for Spring 2009 for Inquiries into Teaching and Learning 1 were available from CRTL and copies were submitted to the course Co-‐Directors. Evaluation findings go to program directors and to chairs of the sponsoring department of the course (in this case Teaching & Learning). The eighth course on a transfer student’s transcript was E03.0001, New Student Seminar, fall 2009, which was apparently not evaluated. The evaluations of courses E25 2037, and E75.1161 – both fall 2010 courses – had not yet been processed by CRTL at the time of the internal audit. They should be available by March 1, 2011. Probe 6. Assess the overall transcript to date in terms of its match with program requirements and guidelines. Discuss major discrepancies if any with an advisor or program area faculty member. QCS elements probed:

• Program Area Directors • Staff and faculty advisors • Program requirements as listed on website

Findings: One auditor assessed a BS in Music transcript, compared it to program guidelines, and met with the student’s Advisor. The student’s transcript indicates that she has completed the courses listed in the “BS Music plan of study.” This auditor also assessed a BS Childhood/Special Ed. Transcript, compared it to Childhood/ Special Ed. program guidelines, and spoke to the student’s Advisor. Student is on track for completing course requirements. Advisor is very familiar with student and clears her for registration each semester. Another auditor compared a graduate transcript in Art and an undergraduate transcript in Educational Theatre to program requirements. In Art, all the courses listed on the website were taken by the student. In Ed. Theatre, the student was missing four courses. The Advisor’s explanation was that the webpage needs updating to reflect changes approved by the faculty. Indeed, the changes were found on the printed program requirements handout, dated 11/4/09. A third auditor noted that a Science MA transcript had discrepancies. The student Advisor and program director explained them as the being the result of stretching requirements over a four-‐semester period or longer for part-‐time students. A fourth auditor noted several apparent discrepancies in a BS Science student’s transcript. The student seemed to be missing the following: 2 natural sciences courses, Integrating Media and Technology into the K-‐12 Curriculum (E19.2018), English core: non-‐Western lit and English core: speech, drama, media. The student’s Advisor explained that his two natural science courses were Nutrition & Health (taken in Steinhardt) and


77

Sound and Music (a physics elective); that his non-‐western lit requirement was appropriately met by taking a literature course called Guilt, Desire & the Law; that his speech, drama, and media requirement was met by taking a Shakespeare course; and finally that he was not required (by grandfathering) to take E19.2018, a recent addition to program requirements.

A fifth auditor reviewed the transcript of a student who started NYU in fall 1996 and noted that it was difficult to assess the overall transcript to date in terms of its match with program requirements and guidelines. The student failed numerous courses and took a leave of absence for multiple semesters. Course titles might have changed in that time and it is likely that requirements changed too. Probe 7. Choose the second course listed on the transcript and verify that it was approved by the relevant faculty committees. QCS elements probed:

• Program Area Directors • Assistant Director for Curriculum Development

Findings: The auditors found that the courses probed were approved by the relevant faculty committees. For E27. 0005, the Assistant Director for Curriculum Development printed out the course description from the course database, and approval was noted in the document. For E03.0002, the course was approved by the Committee on Courses and Programs (CCP) as verified by the Assistant Director of Curriculum Development. For E23.2129, the course number changed on 11/07/96, and the title changed on 4/01/96. Changes to titles, points, and descriptions are handled administratively and require the approval of the chair. Changes may be modest (updating language to reflect current nomenclatures, adding or removing topics covered) such that the course remains faithful to the original focus approved by the department curriculum committee and the school-‐wide Committee on Courses and Proposals. The Assistant Director for Curriculum Development approves course proposals and course changes, pursuant to approval by the CCP. Probe 8. Ask to see the student’s Apprentice system file which tracks field work and field-‐related evaluations. QCS elements probed:

• Office of Clinical Studies • Center for Research on Teaching and Learning (CRTL)

Findings: A Student’s Apprentice file was provided by Office of Clinical Studies. Student Cooperating Teacher’s and Supervisor’s evaluation information was available there also. For fall 2010, the student gave an excellent rating to the Cooperating Teacher and a good rating to her Supervisor. For a second student, the fieldwork placement assignments were noted by semester and student completed ETFQ, with results of student feedback provided. The Supervisor did not submit a DRSTOS evaluation for this student. An auditor met with the staff from CRTL and was informed that the completion of DRSTOS forms is tracked by CRTL. Students who are missing forms either have untrained supervisors or were with trained supervisors


78

who did not submit forms at all. CRTL follows up with these trained supervisors until at least a month into the next semester, with reminders sent roughly every two weeks. Probe 9. Did the student have an educational experience abroad? If yes, how was the course organized? If not, how would the student be informed of the opportunity to study abroad as part of his/her program? QCS elements probed:

• Program Area Directors • COPs • Assistant Dean for Academic & Global Programs

Findings: Four auditors probed these questions. Auditors interviewed the Assistant Dean for Academic and Global Programs, who stated that extensive information is available to all Steinhardt students about study aboard. Graduate students receive email invitations to all information sessions about courses offered. The Dean also works with all academic advisors to offer workshops about courses offered. Undergraduates receive information from academic advisors, promotional brochures, direct emails, word of mouth, and a session of the weekly new student seminar. Auditors noted that undergraduate students’ transcripts identify study abroad course. However, transcripts are not an obvious guide to identifying study abroad when it comes to graduate students, since students may take courses with the same numbers whether taught in NYC or South Africa. The Assistant Dean for Academic & Global Programs indicated that course sections 98-‐99 indicate study abroad. In online versions of the transcript, however, these section numbers are not visible. Probe 10. [For undergraduates] Compare the content courses – in teaching field and liberal arts – to that of a comparable Arts & Science major in the same field.

QCS elements probed:

• Program Area Directors • Staff and faculty advisors • Faculty COPs • Program requirements listed in webpages

Findings: The auditors noted that the cross-‐school comparison is challenging, not only because questions of purpose are involved, but also because the formats for presenting requirements are different. Here, for example, is the description of Steinhardt requirements for English Education: http://steinhardt.nyu.edu/teachlearn/english/bs/program_of_study


79

Here are two web pages that present a comparable description of the CAS English major: http://english.fas.nyu.edu/object/english.1012.ug.req http://english.fas.nyu.edu/object/english.ug.courseinformation The latter involves 10 4-‐credit courses beyond the basic Morse Academic Plan courses (several of which involve English content). Four of the 10 are required (Literary Interpretation, Brit. Lit I and II, and Amer. Lit I), 2 are restricted electives (among 9 courses in critical theories and methods, and among 20 courses in British literature before 1800), and 4 are open electives. Total of 40 credits. The English Ed. Major, by contrast, takes 5 4-‐credit English courses at CAS beyond the Morse plan, of which 4 are restricted electives (including Brit. Lit. and Amer. Lit.), and 1 is an open elective. He or she also takes 6 3-‐credit Steinhardt courses that combine content and pedagogy – for example, The reading of Poetry, and Literature as Exploration. All of these are required. Total of 38 credits. As for comparing liberal arts requirements between CAS and Steinhardt teacher education undergraduates, this requires even further webpage digging on the CAS side. See http://map.cas.nyu.edu/page/abouttheprogram. CAS requires only 3 specific courses – Writing the Essay, Texts and Ideas, and Cultures and Contexts -‐ while Steinhardt requires these 3 plus 2 others – The Advanced College Essay and Expressive Cultures. Moreover, CAS is more liberal in allowing departmental substitutions for the traditional Morse requirements (Quantitative reasoning, and Natural Science I and II). Both schools require two semesters of foreign language study

Faculty

Probe 11. Choose the seventh course listed on the transcript and trace the appointment process of the instructor. In the process, obtain a copy of his or her CV. QCS elements probed:

• Dept. Chairs • Director of Faculty Affairs • Faculty COPS • Director of Human Resources

Findings: Four courses were probed. One auditor met with the Director of Faculty Affairs who provided the instructor’s CV for E85.2139 and reviewed the instructor’ appointment process. The instructor is the Director of Music Education. He was recruited by a faculty search committee in accordance with procedures laid out at the following web address: http://steinhardt.nyu.edu/steinhardt/faculty_affairs/pdfs/faculty_search_guidelines.pdf For E27.2999, the auditor interviewed the instructor. The instructor had been a full-‐time staff member on an external grant as well as a member of the adjunct faculty for 4 years when the Chair of the Department of Teaching & Learning requested in the spring of 2010 her appointment to


80

the full time faculty at the rank of Master Teacher. The Deans met to discuss this request, and forwarded it with the instructor’s CV to the Provost who approved an offer of appointment. The instructor then met with the Director of Faculty Affairs to discuss and accept the offer. CV is in the internal audit folder. For E85.0092, Collegium & Program Seminar, an auditor spoke to the instructor. Since 2007, when the student took the course, the course number has changed. Today it is E80. 1500. The instructor was hired in 2002 by NYU as a Clinical Assistant Professor of Music Business, a title and full-‐time rank she retains today. Her CV is in the internal audit file. For E27.0005, the instructor is the Director of the Office of Clinical Studies. The Director’s CV is on file with the Director of Human Resources, Office of Administration and Finance. He was interviewed for the position by an interview committee and was appointed as an administrator.

Probe 12. Determine the number of tenured or tenure-‐track faculty in Steinhardt who have instructed this student. Trace the appointment and promotion process of the last one listed. QCS elements probed:

• Dept. Chairs • Director of Faculty Affairs • Faculty CVs submitted annually

Findings: Auditors probed 5 transcripts. For the first transcript reviewed, the student took 17 courses in Steinhardt (9 taught by FT faculty and 8 by PT faculty). Two of the 9 FT faculty are tenure-‐track or tenured. The last faculty member listed was appointed to the Department of Teaching and Learning in 1/1/06 as Associate Professor in Mathematics. This faculty member was recruited/ hired/mentored according to guidelines provided by the Director of Faculty Affairs. For a second student, the last tenure-‐track faculty member listed was appointed in 1980 as Associate Professor to the Department of Educational Psychology and in 1990 as Professor, Department of Teaching and Learning, and is currently Professor of Special Education. CV was provided by Department of Teaching and Learning. For a third student, of 24 instructors listed on the transcript since fall 2009, the student was instructed by 2 tenured faculty members. The last one listed is a Professor of Childhood Education. This faculty member was appointed on 9/1/98 as Visiting Professor and on 9/1/99 as Full Professor. Another student’s transcript indicated that he was taught by 2 full time tenured professors and 10 adjunct instructors. The last tenured professor listed, is an Associate Professor with expertise in linguistics, appointed on 9/1/79 as Assistant Professor and on 9/1/85 as Associate Professor.


81

The fifth student has been taught by 3 tenure or tenure-‐track faculty. The last tenure-‐track professor noted is a Professor of Mathematics in her first year of assignment to NYU. The Department of T&L provided the job posting for the position. An Auditor interviewed the Director of Faculty Affairs who discussed the faculty evaluation process that the above referenced faculty would have adhered to. Personnel reviews are conducted annually through a peer review procedure established by the department and by the department chair. Each department has developed criteria for the evaluation of performance by tenured, tenure-‐track, and clinical faculty, master teachers, teachers, and those holding other term appointments. Probe 13. [For students who have completed at least one student teaching assignment] Determine the candidate’s field supervisor, and obtain a copy of his or her CV. QCS elements probed:

• Dept. Chairs and Administrators • Director of Faculty Affairs • Office of Clinical Studies

Findings: Auditors consulted the records of the Office of Clinical Studies to determine the names of the adjunct supervisors assigned to work with five students and they later obtained the CVs of the adjunct supervisors from the Administrator of the Department of Teaching & Learning who processes adjunct appointments for the department. An auditor also consulted the Music Department to determine the name of a supervisor for E85.1048, and obtained the CV of the faculty supervisor (a full-‐time faculty member) from the Office of Faculty Affairs. In the process, the auditor learned that records of Music Ed. and other arts supervisors are now maintained with other area supervisors in the Office of Clinical Studies. Probe 14. Select a course taken during the fall 2010 semester. How was the faculty member recruited, mentored, supported, evaluated? QCS elements probed:

• Dept. Chairs and Administrators • Director of Faculty Affairs • Program Area Directors (Math, English, Childhood, etc.) • Office of Clinical Studies

Findings: Four courses were probed. For E29.2002-‐ Linguistic Analysis-‐ an auditor consulted with the Department of Teaching & Learning and was


82

informed that the instructor was an adjunct professor who has been teaching Linguistic Analysis since 1993 in the Department of Teaching and Learning. He has a PhD in Linguistics, and is reappointed on a semester basis by the T&L Dept. He is invited to participate in MMS COP meetings, he is observed by faculty in MMS and course evaluations are on file in CRTL. For an Inquiries Course, another auditor interviewed the course co-‐director and learned that the co-‐instructors are both adjunct instructors (one a doctoral student and the other a teacher in the school where the course section meets). Both were interviewed by the program co-‐director, following nominations by T&L faculty and the school principal, and appointed by the T&L Department. All course evaluations are in place. Inquiries instructors meet monthly in COP, and are visited once per semester by one of the program co-‐directors. For E17. 2113, the course is taught by a Clinical Assistant Professor in Educational Theatre appointed on 9/1/10. This faculty member was originally appointed as a Teacher on 9/1/02 in the Ed Theatre Dept. As a new faculty member, from fall 2010-‐spring 2011, he participated in new faculty orientation sessions and mentoring workshops. His performance evaluation is available in the office of the Director of Faculty Affairs. For E25.1103, Introduction to Early Childhood and Special Education, the instructor is a full-‐time visiting assistant professor appointed on 9/1/10 with a one-‐year contract. This faculty member is participating in mentoring coordinated by the Associate Dean of Academic and Faculty Affairs and by the Department Chair and submitted yearly evaluation. An auditor spoke with the Director of Faculty Affairs who provided links to Steinhardt websites that detail procedures on faculty searches, and personnel review timelines. Visiting professors are hired at the department level, based on a request submitted to the deans. Some departments post positions on NYU website. An auditor spoke also with the department chair for background on this particular hire. The chair responded that his predecessor had requested an emergency hire in Early Childhood, which had been approved by the Dean as a visiting position (renewable for no more than 3 years). The candidate hired had been an NYU doctoral candidate in this area. The Dean approved the appointment once the candidate’s dissertation had been successfully defended.

Candidates

Probe 15. [For graduate students] Obtain a copy of this student’s admission materials and verify that he or she completed the program’s required content courses prior to admission or, if not, that exceptions were explained and recorded. QCS elements probed:

• Office of Graduate Admissions • Registrar • Associate Dean for Planning & Communication

Findings:


83

An auditor consulted with both the Director of the Office of Graduate Admissions and the Associate Dean for Planning & Communication. They demonstrated that a file is maintained for every student applicant admitted per certification pattern , and that this particular candidate met core content requirements for graduate admission. Probe 16. Contact this student’s staff advisor and ask for evidence of contact and advising. QCS elements probed:

• Dept. student advisors (staff and faculty)

Findings: An auditor spoke to an art student’s advisor. The advisor has a file for the student in her office. The file includes her original application, recent transcript, student teaching hours log, a copy of the course sequencing form for tracking progress, and past email communications about various course registration questions. The same pattern evolved when interviewing advisors assigned to other teacher education program areas. Concern was raised when tracking the advisement for a student who is enrolled in a new program area that is a collaboration of the French Department and the Department of Teaching & Learning, and involves study at NYU/Paris and NYU/Washington Square. Tracking her advisement presented a data challenge because of the joint nature of the program, and – according to the T&L advisor -‐ limitations on inter-‐school data exchange, and relatively open course options which require more intensive advising. Probe 17. [For students who have completed at least one student teaching placement] Find the record of this student’s DRSTOS-‐R assessment. QCS elements probed:

• CRTL Findings: Four students’ DRSTOS-‐R assessment records were requested by the auditors. No DRSTOS-‐R assessment was available for two of the four because the supervisors were not DRSTOS-‐trained. The other 2 students had DRSTOS-‐R assessments on file, and these were presented to the auditor by CRTL. Probe 18. Take the transcript to the Teacher Certification Officer and verify that the student is making normal progress toward meeting certification requirements.


84

QCS elements probed: • Teacher Certification Officer

Findings: Auditors met with the Certification Officer to review progress made by four students towards meeting certification requirements. For the first student, the Certification Officer stated that the student graduated 1/24/2011 and passed all exams. However, the student is not certified to teach. In probing further, the Certification Officer noted that the student had not yet completed the application for certification. For the remaining transcripts, the Certification Officer stated that the students had not yet taken the certification exams. The Certification Officer also noted that not all students apply for NYS Certification. Some plan to move to another state or country. Probe 19. If this student had a learning disability or a handicapping condition of any kind, what services would he or she be afforded? Obtain evidence of their availability. QCS elements probed:

• Director of Student Services • Program Area directors • Moses Center website

Findings: All auditors probed this question. This probe applied for graduate and undergraduate students whose transcripts were examined. Auditors consulted a cabinet member of Teaching and Learning, the webpages of the NYU Moses Center serving students with disabilities, the Dean of Student Affairs and the Director of Student Services. The latter explained that disability services are a university-‐wide service not a Steinhardt service. The NYU policy is that any reasonable accommodation will be made for a student with disabilities – for example, access to adaptive technologies, sign language interpretation and assessment adjustments in terms of timing and settings. These require students’ willingness to self-‐identify and provide back-‐up documentation – for example, medical or psychological. When students apply for services, a committee composed of Deans and staff from admissions reviews the application for determination of appropriate services.

Probe 20.Track the admission of this student in terms of the process followed and the decision makers involved.

QCS elements probed • Office of Graduate Admissions • Associate Dean for Planning & Communication

Findings: An auditor met with the Associate Dean for Planning and Communications who explained that undergraduate admission to all NYU schools is screened centrally by NYU Admissions. In the case of the childhood majors probed, the admissions staff would have looked for


85

experience working with children, leadership, and evidence of academic engagement as well as academic accomplishment. 2-‐3 readers review all aspects: letters, essay, test scores, transcripts. Two additional auditors consulted with the Director of Graduate Admissions who answered the probe as follows for a TESOL major: 11/2/09: Candidate submitted an online application via the Embark application system. The student applied to the Teaching a Foreign Language 7-‐12 and TESOL dual-‐certification program (FLTS) for the spring 2010 semester. Admissions staff decided to admit the student, though they also decided in terms of core content requirements that she was deficient by one science course. 11/16/09: Admission letter mailed to candidate along with a form indicating the science deficiency. 12/09: Documents were submitted from the Department of T&L indicating a program change from FLTS to TESOL all grades (TSOG). 1/11/10: Admission letter mailed to candidate confirming major change from FLTS to TSOG, and also reiterating the science deficiency. Probe 21. Determine the career advisement available to this student. QCS elements probed:

• Associate Dean and Office of Student Affairs • Dept. student advisors (staff and faculty) • Dept. Student Life Committee (T&L) • Office of Clinical Studies

Findings: Auditors reviewed webpage for the Wasserman Center for Career Development (www.nyu.edu/careerdevelopment), including NYU CareerNet, and NYU Steinhardt Office of Student Affairs webpage (steinhardt.nyu.edu/counseling/career) and found useful information. Via interviews with NYU Partnership Coordinator, auditors noted that workshops are scheduled regularly by the Department of Teaching & Learning involving mock interviews, as well as job fairs where prospective employers are present. Faculty also provide feedback on resumes. An Advisor in T&L regularly posts job openings in messages sent to all students. The Office of Clinical Studies has also recently cultivated relationships with charter school management organizations, and provides names and addresses of these to students, as well as names and addresses of new schools likely (because of their growth) to be adding teachers even under current job freeze conditions in NY. Besides the Wasserman Center and the Office of Clinical Studies, the T&L Student Life Committee has also been active in this area. There is an active list-‐serve maintained by Department of Performing Arts Professions and career information is sent to students on a daily basis (email documentation is provided in auditor’s files). The Assistant to Chair of the Department of Teaching & Learning maintains a list serve with job postings-‐open to current undergraduates, graduates, and alumni for up to 2 years. Probe 22. Determine the post-‐graduation mentoring available to this student from NYU if he or she enters teaching.


86

QCS elements probed: • Teacher Certification Officer • Office of Clinical Studies

Findings: An auditor interviewed the Certification Officer. He provided a handout describing a new NYS requirement that new teachers have mentors, but this is an obligation of the employer not of NYU. For several years, however, this auditor was associated with NYU’s own mentoring program – first, the Early Career Support Network (ECSN) funded by the Booth-‐Ferris Foundation, and the Early Career Project funded by the Wachovia Foundation and the Petrie Foundation. This provided regular after-‐school, school-‐based mentoring to NYU graduates at four partner schools located near concentrations of alumni. They featured advice from NYU faculty and partner school faculty in such areas as classroom management and integration of students with disabilities. However the program ended when the external funding ended.

Probe 23. When will this student take the NYS ATS-‐W, the LAST and the Content NYS exam? What is the process of monitoring the student progress towards meeting these requirements for certification? QCS elements probed:

• Associate Dean and Office of Student Affairs • Teacher Certification Officer • Program web pages

Findings: Three auditors probed this question for three students and met with the Certification Officer. For the first student, the Certification Officer stated that the student has not taken any state exams yet. He also identified a variety of sources of information about certification provided to students (student teaching seminar, certification workshops, individual appt. with Certification Officer, fieldwork course, handouts, webpage as well as the tests that students sit for). The second student had not completed the state certification tests. The Certification Officer pointed out, however, that he cannot monitor whether candidates have registered for the test (done individually online) – only whether they have taken and passed exams. Their scores are collected and analyzed by the CRTL, and their progress in preparing for the tests is part of the overall advising function of the department faculty and staff. The third student, according to the Certification Officer, had not taken any certification exams. However, this is to be expected as the student is only halfway through his program of study. He has not yet started his student teaching. More about the process of informing


87

and monitoring the student’s progress towards meeting these certification requirements can be found in a memo from the Certification Officer (in audit file). Probe 24. [For undergraduates] Verify that the student is on track to complete degree requirements in terms of both content and pedagogical requirements. QCS elements probed:

• Associate Dean and Office of Student Affairs • Teacher Certification Officer • Other (Steinhardt website)

Findings: Auditors probed this question for six students with the following results: A review of an Ed. Theater transcript in light of requirements stated on webpage, found two discrepancies which a faculty member in the program identified as a problem of outdated web information rather than lack of student progress in meeting actual requirements. A second Ed Theatre transcript examined provided a complex Ed. Theater case. Student has interest in working with special populations. Also did spring study abroad and has elected a minor in Studio Arts requiring 16 additional credits. An auditor met with the Director of Undergraduate Studies for Ed. Theatre who is the student’s advisor. She reported that she monitors, negotiates, and formalizes each substitutions to ensure that the student remains on track for graduation within a 4-‐year term. She pointed out that all students have to be cleared for registration by advisors, a twice-‐yearly formal check on progress. For a BA Science major, there were some discrepancies between courses listed on this transcript and the program’s website, though the student has now graduated. See auditor’s folder for website material. The program director accounted for these as legitimate substitutions approved by the student’s advisor. A review of an early childhood transcript confirmed that a student is on track to complete degree requirements. Requirements were confirmed by program director who also examined the transcript. An auditor looked also at the science courses required for a science education major. Student appears to have taken (or is currently registered for) all science courses needed for the degree. However, the student obtained a D in Organic Chemistry II. The auditor checked with the student’s advisor who explained that students need an overall 2.5 GPA in order to progress to student teaching, but that grades under C raise warning flags for advisors. Typically, if a student gets less than C, the advisor will find out why and consult with the program advisor who will also intervene. This student’s advisor is no longer at NYU, however, and his current advisor could find no note in the student’s file concerning the low chemistry grade. While the new advisor acknowledged the advisability of a paper trail, she also said that there is no policy expressly prohibiting a student with one such grade from progressing toward graduation, so long as the GPA does not drop below 2.5.


88

Resources

Probe 25. Determine the median course listed on the transcript. Find out the number of students assigned to the course and visit the room where it met in order to assess its appropriateness in terms of space and access to technology. QCS elements probed:

• Associate Dean of Administration and Finance • Campus Media • Department Administrative Assistants • Department Chairs • NYU webpages

Findings: A total of 12 courses were probed, with the following results:

Course Title Number of Students registered

Location/Room Max. no. of students accommodated

Access to Technology

E92.2272 (fall 2010)

Art Education 17 Barney, 204 22 Permanent media equipment installed

E63.0023 spring 2009

Human Dev. II: Early Adolescents

33 Silver, 207 125 Equipment/internet access available

E27.1030.001 (spring 2011)

Lang. Acquisition and Literacy Educ.

9 Tisch, LC5 20 Media equipment available/internet access

E11.2521.001 (spring 2010)

Lit & The Adolescent Experience

20 Waverly 435 33 AV services available

E29.2206.002 (spring 2011)

Second Language Theory and Practice

18 Silver, 706 47 Permanent media equipment installed.

E17.2193 (summer 2010)

Drama in Education I

21 Silver, 410 37 Full audio visual services available

E26.2001 (fall 2010)

Language and Literacy in the

20 Silver, 514 42 Appropriate AV Support


89

Early Years E29.2040 summer 2010

Teaching Second Language in a Technological Society

20 194 Mercer, 304 30 Data/Video Equipment/internet

E85.1068 (spring 2009)

Music History 136 Loewe Theatre 297 Adequate AV Capacity

E12.2101 fall 2010 Professionalized Subject Matter in Mathematics

7 194 Mercer, 308

25 Adequate AV Capacity

E11.2501 (fall 2010)

Masters Seminar in English Education

20 48 Cooper Square, 118

20+ No windows, large column in middle. AV equipment available/ Campus Media does not support this location

E14.2052 spring 2010

Field study in ecology

15 Meyer, 105 20 Technology and wireless internet available

Probe 26. Pick any program area course listed on the transcript. Obtain the syllabus for the course. Check the availability of two sources referenced in the course within the NYU library system. QCS elements probed:

• Department Administrative Assistants • Bobcat – library online catalog

Findings: Four auditors probed this questions and reported the following:

Course Title Syllabus Sources Referenced Availability in NYU Library System E17.1006 Intro to Theatre for Young

Audiences No syllabus available

N/A N/A

E23.1135 Trends/Prob./Secondary Soc. Studies Education

Available 2 required texts Both Available

E26.1176 Language Reading Instruction in early childhood

Available 3 required texts One text available in an earlier edition

V89.0010 Statistical Reasoning for the Behavioral Sciences

2009 syllabus available

1 source referenced Available


90

E12.2115 Teaching Elementary Math Available 2 textbooks One available in an earlier edition E29.2201.002

Teaching Elementary Math/Second Language Classroom

Available 2 textbooks One available in in the correct edition

E11.1600 Integrating Reading & Writing with Adolescents.

Available 2 textbooks 5 supplemental sources

One text available in a different edition. Of the 5 supplemental sources, 1 is the right edition, a 2nd is available in another edition. 3 supplemental sources were not available.

Probe 27. Assess the transcript – and other sources as necessary – for evidence that this student has experienced the use of inventive technologies for teaching and learning. QCS elements probed:

• Department Administrative Assistants • Other (course syllabi)

Findings: Of 6 transcripts reviewed, 5 students had experienced courses using inventive technologies. The transcript of an MA Ed. Theatre Student did not list a technology course. In checking courses on the transcript, however, it was noted that the student had enrolled in E27.2999, the Social Responsibilities of Educators. This course is a blended course, or face-‐to-‐face plus online via blackboard and VIMEO. [The internal auditors’ reference to “blended” here is to the fact that this is a technology enhanced conference-‐based course. It is not delivered in a distance format. All students are on campus.] A second student was enrolled in E11.2511, which also uses Blackboard. A third student completed E78.2029, Teaching Resources for Performing Arts. The syllabus indicates use of a range of creative technologies. A fourth student is on track to take E25.1124, a required 3-‐credit course in the spring senior semester of the Childhood/Special Ed. undergraduate program. Course is called Technology in Childhood Education. The fifth student enrolled in E75.2161, and the course makes extensive use of Blackboard. For one student, the transcript had no traceable references to technology.

Probe 28. Determine the proportion of full-‐time versus part-‐time faculty in Steinhardt who instructed this student (as well as tenure-‐track


91

versus non-‐tenure-‐track full-‐time faculty). QCS elements probed:

• Associate Dean of Administration and Finance • Department Administrative Assistants

Findings: N.B. For the undergraduates, the auditors assessed only Steinhardt Courses because it was not possible to assess the instructional status of CAS instructors. Auditors probed 11 transcripts and assessed Steinhardt courses as follows:

Student Major Total No. of Courses Taken

Full Time Faculty Adjuncts Administrator No. and % (Clinical Office)

Student A TESOL (Graduate) 14 5 ( 36%) 9 (64%) Student B Childhood Ed. (Undergraduate) 19 14 (74%) 5 (26%) Student C English Ed. (Undergraduate) 16 6 (38%) 10 (62%) Student D Early Childhood/Special Ed.

(Undergraduate) 18 5 (27%) 10 (56% 3 (16%)

Student E Dance (Graduate) 24 6 (33%) 16 (67%) Student F English Education ( Graduate) 14 3 (21%) 10 (71%) 1 (7%) Student G Childhood/Bilingual

(Undergraduate) 29 12 (41%) 17 (59%)

Student H Childhood/Special Ed. (Undergraduate)

18 5 (28%) 13 (72&)

Student I Childhood/Special Ed. (Undergraduate, transfer)

28 8 (29%) 20 (71%)

Student J Social Studies (Graduate) 18 6 (33%) 10 (56%) 2 (11%) Student K Math Education (Graduate) 11 4 (36%) 6 (55%) 1 (9%)


92

AUDIT RESULTS & DISCUSSION In their final meeting on the afternoon of February 2, 2010, the Internal Audit Team members did two things. First, they agreed on a set of what they called Summative Findings and Recommended Action Steps. These are listed below. Then for each probe and its findings, they posed questions for the faculty. Over the next two weeks, as additional findings came in, they expanded the list of questions accordingly.12 The questions – with the backdrop formed by the Summative Findings and Recommended Action Steps – has been the basis of a continuing process of self-‐inquiry that continues today in what is now a strengthened Teacher Education Working Group (TEWG) (as per one of the recommended Action Steps). The Questions and TEWG-‐generated answers are listed below in Table A.1. Summative Findings The internal auditors’ overall summative conclusion is that the Quality Control System is working as intended, and does ensure the quality of the program itself and the quality of student learning. Furthermore, the auditors concluded:

• that they were impressed by the accessibility of faculty and administrative staff and the data and information systems available to answer most of their questions;

• that, in general, they found encounters with the people who manage the QCS to be very positive; • that the faculty and administrative advisors knew their students well, and were able to give

quite detailed accounts of the students’ academic progress and choices, and answered questions thoroughly;

• that administrators in the Office of Clinical Studies were knowledgeable and that the auditors came away with a good understanding of students' field work;

• that, as probed, the student information systems (Apprentice, Albert, and SIS) are working; • that some weaknesses in the QCS emerged in specific instances when the auditors tried to find out

where information is stored and how to get at it. Auditors’ Recommended Action Steps The following recommendations were made on the afternoon of the final day of the Internal Audit, before the questions raised by the auditors were fully formulated, and certainly before the faculty and Deans could respond to them. The main purpose of the recommendations was to capture the auditors’ immediate and overall impressions, and in the process to inform the faculty’s answering of the questions and their discussion of the specific findings. Curriculum

• If faculty communities of practice are critical quality control mechanisms for curriculum – which seems likely – then their presence and membership ought to be more easily discernible. For example, they should be more present on the web in the form of public pages as well as listservs and blogs. Their meeting times and meeting minutes should be posted. This would enable best practices to spread, and enable the faculty to see what parts of the overall program may be lacking strong communities of practice.

12 In all, the auditors posed and TEWG answered 53 questions. However, in this final draft of the Brief, these have been reduced to 38 by way of eliminating redundancy and focusing only on matters clearly relevant to the QCS. The entire Q & A set is, however, available in the TEAC Coordinator’s office.


93

• Syllabi are obviously critical quality control mechanisms – from the point of inception of a course to the latest delivery of it. But we found it sometimes hard to obtain syllabi – for long established courses, and also current versions of courses. We also heard that department administrators find it hard to keep their archives of syllabi up to date. We think that a culture of online syllabi (and also CV) storage is needed – making them available on public webpages.

• The clinical studies portion of NYU teacher education is massive and sprawling – across many course numbers and supervisory arrangements. Consequently, the quality control mechanisms are complicated too – some of them housed in the Office of Clinical Studies, some in departments, some in the Center for Research on Teaching and Learning. Some procedures are well documented – for example, through Apprentice, or in handbooks – and others not. At a time when appreciation of the importance of the clinical side of teacher education is growing at NYU and beyond, and in the wake of improvements made at NYU in this area in recent years, this may be a good time to take stock of the clinical systems overall. Where can greater efficiencies be obtained? Where do procedures need more documentation? How can available data be better utilized? Where can stronger connections be made between field and course? And so on.

• The Steinhardt website is sprawling too, and we found older versions of course planning sheets, course numbers, and the like on the website. How can these be culled more efficiently on a continuing basis?

• Qualification to administer DRSTOS has to expand from the current 85% to 100%.

Faculty • There should be a larger role in the program for TEWG, or for some other central coordinating body

of the faculty. There is great value in keeping teacher education at NYU very bonded to content-‐focused faculty groups, but in some respects the overall program seems to lack a center. This manifests itself most clearly in two areas. First is the role of clinical studies, as noted above. The other has to do with data systems, data management, and data use.

• We found the management of faculty CVs very uneven across departments. Again, online may be the way to go with this.

• The distribution of faculty expertise in terms of clinical and tenure-‐track needs to be evened out – across undergrad and grad teaching.

• There should be more collaboration across faculty groups – both within and across departments. There is currently a wealth of information and good practice that could be shared across the faculty, but may not currently be shared.

Candidates

• As the University introduces Peoplesoft as a data management system, there seems to be a great opportunity to create new efficiencies and deal with current inefficiencies.

• How can NYU increase the rate of return for the End-‐of-‐Term Feedback Questionnaires, and ensure that the resulting data on cooperating teachers and supervisors is used?

Resources

• NYU needs to pay more attention to the role of technologies in teaching and learning, as well as in data storage and retrieval. This will require new resources (for example, for campus-‐based teacher education courses, and for data systems).

The Internal Auditors highly recommend that NYU Teacher Education at least every other year conduct an internal audit on the same design as the 2011 Audit.


94

Auditors’ Questions and TEWG-‐Generated Answers Between March 28, 2011, when the Teacher Education Working Group began to deliberate on the questions raised by the Internal Auditors, and April 29 when it completed a penultimate draft of this Appendix, TEWG members, Teacher Education Council members, Deans of Steinhardt and of the College of Arts and Science, and other faculty and staff worked to formulate responses to a very insightful set of questions. This formulation – processed in face-‐to-‐face meetings and interviews, phone calls, emails, and other communication – constitutes the discussion of findings reported in Table A.1. It was a fruitful start to a longer discussion that has continued in the fall 2011 semester – and that will likely prove instructive to NYU’s long-‐term exercise of Quality Principle II (faculty learning and inquiry). By way of affirming the great value of the 2011 Internal Audit, the Teacher Education Working Group began this discussion of findings by endorsing the recommendation of the auditors to conduct an internal audit every two years, and on May 4, Dean Brabeck committed to support a biennial internal audit. Moreover, in response to the auditors’ first recommendation concerning faculty, the Dean asked the Department of Teaching and Learning to choose three additional tenured professors to sit on TEWG (thus making the overall faculty members of the group roughly proportional to the departmental spread of the teacher education faculty as a whole), and she asked the Chair of TEWG to ensure a strong connection between TEWG and the Teaching & Learning Curriculum Committee (the chair of the latter is now an ex officio TEWG member) and between TEWG and the Office of Clinical Studies. Resulting changes are likely to ensure a more central role for TEWG in NYU teacher education as the auditors recommended, while at the same time continuing the historically prominent role played by program area COPs across departments with respect to content curriculum and management, and by the Department of Teaching & Learning with respect to core curriculum.

Table A.1 TEWG-‐Generated Answers to Internal Auditors’ Questions

Curriculum Q1. How wide is the variability in terms of the meeting practices of communities of practice (e.g. frequency and documentation practices), and is the variability appropriate or not? A. The Teacher Education Working Group (TEWG) – working with the Center for Research on Teaching and Learning -‐ will conduct a survey of teacher education communities of practice in fall 2011 to determine an answer to this question and other questions explicitly or implicitly raised by the Internal Audit – for example, regarding the size and composition of a COP, its visibility, its awareness of both campus-‐based and school-‐based dimensions of NYU teacher education, and its grounding in research and best practice. In terms of the specific question here, anecdotal evidence suggests that there is significant variability in COP meeting practices. What matters most in TEWG’s view is a sense on the part of a COP that it is a community of practice and that as such it has a collective responsibility for the outcomes of practice. It seems likely that frequency and documentation would contribute to the development of such a sense. Q2. Are program faculty communities of practice (COPs) and the program supervisors’ COPs typically separate, and if so, what limitations ensue in terms of quality control? A. This too will be a point of inquiry for the fall survey, though it seems likely that the survey will find significant separation in some program areas and ample integration in others. In general, the NYU teacher education program aims for integration of coursework and fieldwork, and a separation of communities of practice along these lines seems counterproductive. Finding out not only that there is variation but also how program areas manage integration will be very useful.


95

Q3. Would greater public identification of the curriculum communities of practice (e.g. via website) add to their effectiveness? A. Yes. Indeed, there is anecdotal evidence that the mere identification of communities of practice as a key quality control mechanism in the internal audit spurred several COPs to meet more frequently and to have a greater sense of their role. TEWG in collaboration with CRTL will track the development of NYU Teacher Education COPs in its annual TEAC reports. Q4. Are there plans to move syllabi storage into an online format for accreditation purposes, as well as to serve the planning needs of instructors and students? A. Currently many individual full-‐time faculty include their syllabi as part of their biographical data at http://steinhardt.nyu.edu/faculty_bios/list/Faculty/All, but Steinhardt does not otherwise collect electronic versions of syllabi or post them on its website on a school-‐wide basis. Instead, most departments collect paper versions of syllabi and store them; others have created local web-‐based solutions for posting syllabi. The TEAC Coordinators, with the assistance of appropriate teacher education staff, will work with the school’s web team in the fall to create and implement a system for posting syllabi on the teacher education websites. In addition, the TEAC Coordinators – as part of their preparation for the TEAC auditors – will urge all members of the teacher education faculty to attach current syllabi as well as current CVs to their bios. Q5. Do barriers between schools and departments in terms of access to student data interfere with advisement and accountability? If so, how are they typically resolved? A. Evidence from the internal audit suggests that access barriers between departments is a barrier to general accountability, and also to advisement efforts across departments – for example, regarding such joint programs as Foreign Language Education/French, or English Education/Educational Theatre. Both these programs have, however, become skillful in crossing these barriers. The migration of the Student Information System (SIS) into a Peoplesoft platform may make this easier, and TEWG urges NYU to exploit such opportunities. More generally, it sees this problem as evidence of a need to heighten central operational capacity in the NYU teacher education program even while preserving decentralized or program-‐area autonomy in many respects. A proposed change in the organization of faculty responsibilities in the Department of Teaching and Learning may help in this regard in its suggestion that TEWG should expand its faculty representation and its oversight of the core courses and field-‐based elements of teacher education. Q6. How does the analysis of students’ responses to placement quality in any given semester affect subsequent placement options? A. The staff in the Office of Clinical Studies reviews the student teacher evaluations of cooperating teachers and the supervisor evaluations of cooperating teachers, then compares these, when appropriate, to any previous evaluations of the same cooperating teachers. If there are consistent negative evaluations, NYU notifies the school, and takes account of the evaluations in subsequent placement decisions. Meanwhile, NYU’s increasing emphasis on partnership schools, as well as the more recent development of residency options, signals its pursuit of multiple-‐mentorship for its teacher education candidates whereby the weaknesses in one mentor are balanced by the strengths of other available ones within a broader mentoring environment. Q7. Does the online request system for student teaching placement reflect up-‐to-‐date checks on


96

availability? A. The online system does not indicate to students the actual availability of placements at any one school. Schools generally confirm available placements only after students submit their online requests. Q8. Findings suggest that the role of faculty with respect to the student teaching placement process varies across certification areas. How much does it vary (for example, how typical or atypical is the music example), and how does variance affect quality, efficiency, and accountability? A. There has been significant movement towards greater unification of placement procedures and documents across program areas, including the introduction of a single database, Apprentice. There has also been a substantial increase over the last several years in DRSTOS usage across all program areas. However, there remains variation in the involvement of faculty in fieldwork, and thus in the integration of campus-‐based work and fieldwork in the experiences of students. A variety of factors account for the variation including staffing and resources. What seems crucial in terms of making progress in this area is sharing among the COPs. For example, how do those program areas that situate methods and other courses in field settings (Childhood, Early Childhood, Social Studies) manage the details? How do those program areas that handle the bulk of field supervision with full-‐time faculty (Science, Music, Theatre Education) manage the load? Q9. What is the rate of response in supervisors’ evaluations of student teaching placements, and how are these evaluations used? A. The rate of response of the on-‐line surveys for fall 2010 was only 55%. On one hand, there was a slow transition for many supervisors from the verbal feedback through meetings with the Office of Clinical Studies to the new on-‐line survey; on the other hand, there were technical difficulties with the link on the web site, which prevented many supervisors from completing the survey . The link has been repaired and the Office of Clinical Studies expects to increase the formal reporting rate significantly with a goal of 100%. Steps taken include notifying supervisors that it can consider only formally rendered evaluations and that evaluating placements is an important part of their job responsibility. The Office compares the feedback by supervisors with feedback by student teachers to evaluate cooperating teachers. When it identifies consistent negative feedback, it informs the school that the cooperating teacher involved is not a suitable choice for future placements. Q10. Since general pedagogical core courses – as distinct from program area requirements -‐ lie outside the boundaries of program areas and departments by definition, how does NYU assure the quality of their staffing and learning outcomes? A. The faculty has recently wrestled with this question. There is widespread agreement that assignment of general pedagogical core courses to program areas and departments does not by itself ensure that quality will be well tended. What seems key, however, is that a community of practice forms with respect to the course, that it deliberately tends to quality as a regular part of its agenda, and that it is led by one or more full-‐time faculty members. There are such arrangements now in place for several general pedagogical core courses (Inquiries, Social Responsibilities, New Student Seminar, Educating Students with Special Needs, and Fieldwork in Schools and other Educational Settings). Building such arrangements for the others (Literacy and Language Acquisition, Educating Students with Disabilities in the Middle and High School, and Human Development/Adolescent Development) will constitute a major part of TEWG’s agenda in 2011-‐2012. TEWG will work on this agenda in collaboration with relevant program areas – Literacy, English Education, and MMS for the Literacy and Language Acquisition course;


97

Special Education and the secondary program areas for the Disabilities course; and the Steinhardt Psychology faculty as well as Department of Teaching and Learning Human Development faculty for the Human/Adolescent Development courses. Q11. Is the Math Director’s description of current COP efforts there typical of periodical reviews conducted across COPs? [“The Math Program Area Director reported that she and the faculty COP are looking at all the math courses, the sequencing of courses, the requirements, etc., with an eye to developing a more coherent overall pattern.”] A. The review she mentions was triggered by a major turnover in the math education faculty. Other review triggers include policy changes (for example, the re-‐registration of teacher education programs in the late 1990s). However, programs also undergo change for many other reasons, including efforts to address student feedback. For example, Heather Hererra, Assistant Director of Curriculum Development, and a TEWG member, notes that changes were made recently to Early Childhood/Special Ed. and Childhood/Special Ed. based on feedback that some of the courses were redundant. Still other changes are the result of responses to evaluations such as the Levine Study (2006) in which a national sample of students reported that they felt less than satisfactorily prepared to work with technology in the classroom. On the basis of this finding, the NYU teacher education faculty collaborated with the Administration, Leadership, and Technology faculty and developed a new course, Integrating Technology into the K-‐12 classroom. In their recommended action steps, however, the internal auditors urge that general reviews be more frequent and regularly scheduled. TEWG will consider this recommendation, and based on a planned survey of program area COPs in fall 2011, may call for periodic reviews. Q12. Why is a core course like New Student Seminar not evaluated? A. TEWG was not aware that this course has not been routinely evaluated. CRTL will add the course to its evaluation list beginning in the fall of 2011. Meanwhile, the Steinhardt School is implementing a school-‐wide online course evaluation system. Implementation is in the pilot phase through fall 2011 and will launch in full for spring 2012 courses. Under this system, all courses will be evaluated every semester, including those that might not have been included in the past. Q13. Why was a transfer student required to take New Student Seminar in his/her fifth semester at NYU and second semester at Steinhardt? A. There are special sections for transfer students. If the course is taken in any semester other than the first, it generally means that the transfer sections available were at times that conflicted with the student’s required courses. The Steinhardt Website provides the following information on the New Student Seminar (E03.0001):

During their first semester in residence, incoming freshmen and transfer students are required to register for New Student Seminar (E03.0001). The New Student Seminar is a noncredit course given on a pass/fail basis, organized by curriculum to explore professional issues and to provide ongoing orientation and guidance.

Arguably, such a course could be valuable at any time. Q14. What mechanisms exist for ensuring that liberal arts courses taken in NYU schools other than the College of Arts and Science are comparable to those taken within CAS? A. Students may take liberal arts courses within the Steinhardt School to meet designated Morse Academic Plan requirements. The Steinhardt Dean of Faculty & Academic Affairs annually reviews the list of Steinhardt courses which satisfy liberal arts requirements, and requests that full-‐time faculty teach these courses to ensure that undergraduates gain exposure to full-‐time faculty across the curriculum.


98

Instructors of these courses typically have terminal degrees in fields such as history, philosophy, sociology, biology, etc. The Steinhardt liberal arts courses must also be approved at the departmental level, and by the Committee on Courses and Programs. The courses are reviewed for content, level, appropriate course load, learning objectives, and adherence to the New York State Department of Education’s definition of liberal arts:

• Independent of specific application • Breadth and scope in principle covered • Not definitely directed toward particular career or specific professional objectives • Not chiefly "how to" in manipulative skills or techniques • Not "applied" aspects of a field

Note that Steinhardt, while a professional school, also has faculty with arts as well as arts and science practices in teaching and research – for example, within the Department of Humanities and Social Sciences in the Professions, and the Department of Applied Psychology. Even the three departments housing the Teacher Education Program have faculty with substantial scholarly records as – for example -‐ historians, scientists, artists, and musicians. For the most part, however, Steinhardt undergraduates complete their Morse Academic Plan requirements through the College of Arts and Science. For the most part too (except in Art, Music, and Theatre), they complete their content requirements in the College of Arts and Science as electives, and as variations of the College of Arts and Science majors. At its April 26, 2011, meeting, the Teacher Education Council committed to making the question of what should constitute the content knowledge of teacher education candidates at NYU a recurring item on its agendas. Q15. Who approves exceptions to program requirements, and who records and maintains records of these exceptions? A. At the point of admission, program area directors may approve exceptions in program requirements, which are recorded with the particular office of admissions (UG or grad). In some cases, after matriculation, exceptions are granted which are proposed by program area directors, approved by the Dean of Faculty & Academic Affairs, recorded with the Office of Graduate Studies or the Steinhardt undergraduate advisement office, and ultimately agreed upon by the Registrar. Q16. What is the role of departmental curriculum committees with respect to teacher education as an interdepartmental effort of NYU? A. Departmental curriculum committees need to consider department resources as well as how changes to teacher education courses or program areas may impact students in other areas of departmental concern (for example, staffing courses in non-‐certification areas). At the same time, interdepartmental faculty scrutiny is necessary for changes in, and continuing scrutiny of the teacher education core curriculum (including fieldwork). For this reason, the faculty is currently considering a plan to add faculty capacity to TEWG which would then collaborate with departmental curriculum committees on oversight of the core. This is one of several steps under consideration to develop a stronger center for NYU teacher education even while honoring the value and vitality of its de-‐centralized features (e.g., links to the arts faculties and small communities of practice). Q17. Why is the evaluation of fieldwork placements and fieldwork supervision not integrated with the evaluation of other elements of the program – for example, coursework, educational beliefs, and student teaching outcomes?


99

A. Historically, the mechanisms at NYU for evaluating placements and supervisors developed earlier and on a different track from the systematic and schoolwide evaluation of courses at Steinhardt which is relatively recent. The separation has some pragmatic justification (for example, the need to use somewhat different instruments), but it also exemplifies traditional conceptions of fieldwork as posterior to coursework – a matter of application rather than of first-‐hand learning. Clearly, new conceptions of teacher education have emerged which challenge these conceptions, and that therefore urge integration rather than separation. Indeed, NYU is in the forefront of teacher education programs nationally in its embrace of these new conceptions, as exemplified by its partnership schools program, its school-‐based courses, its Learning Partners program, its emerging residency options, and other program features. This question is therefore one that the faculty should address soon, and TEWG will make it part of its fall 2011 agenda. Q18. Study abroad is a major university-‐wide emphasis of NYU (as evident in recent online messages from the NYU President), but how prevalent is it among teacher education candidates? What are the incentives and deterrents to their participation? A. At the undergraduate level, about 20% of undergraduate students in the Teaching and Learning Department study abroad, many of them during spring semester of sophomore year. Steinhardt does not at this point measure teacher education participation across departments. The school-‐wide average (which includes the teacher education portions of the Art and Music departments) is about 35%. The school is working to address the discrepancy between this overall school figure and the Teaching & Learning figure by starting study abroad advisement very early (in fact, at freshman orientation), delineating pathways for including study abroad in the undergraduate degree in teaching, and creating new opportunities for students to engage in related coursework and field experiences while studying abroad. New York State permits prospective foreign language teachers, for example, to obtain credit for clinical experiences in schools where the language they will teach is the language of instruction. Students learning to teach any other program may also do school-‐based fieldwork at any of the three Anglophone NYU sites – London, Ghana, or Australia. Of these sites, NYU-‐London is already prepared to offer pre-‐student teaching supervised field work for undergraduates. Through a recently concluded formal partnership with the University of London’s Institute of Education, graduate students will also gain opportunities to study teacher education there. Upper-‐level undergraduates and graduate students also take short-‐term, faculty-‐led summer abroad courses, such as literacy teaching and learning in London, educational theatre in London and Dublin, and multilingual/multicultural studies in Shanghai. Q19. How might study abroad be made more evident on transcripts? A. NYU has just changed the course numbering system, so now all courses offered abroad at the undergraduate level will have a 9000-‐level number. Graduate courses are marked by course title. Q20. Do faculty members from Steinhardt and the College of Arts and Science consult with each other regarding liberal arts and major content requirements? A. The examples cited by the auditors regarding differences between discipline-‐related majors in CAS and those in Steinhardt are good ones for the Teacher Education Council to consider. Indeed, the Council has begun to wrestle with questions that underlie such differences – for example, Should a childhood education major have a single-‐focus disciplinary arts and science major, or an interdisciplinary one? Should the English major and English Education major have the same study opportunities and demands with respect to English or not, and if not why not? At its meeting on April 26, when it reviewed findings


100

from the internal audit, the Teacher Education Council decided to make the question of the Childhood major’s content requirements a priority for discussion in the fall of 2011, and to take up other such questions as a recurring agenda item. The Teacher Education Council affords a great opportunity to explore such questions. It is co-‐chaired by the Dean of the College of Arts and Science and the Dean of Steinhardt, and is comprised of equal numbers of Arts and Science faculty on the one hand, and Steinhardt teacher-‐education or teacher-‐education-‐related faculty on the other hand. It meets only once per semester, however. Thus the Council co-‐chairs have urged members to meet informally too – within discipline-‐alike groups. Recently, by means of such meetings, the Council has achieved notable success in planning and launching joint programs in French and Spanish, and a new educational minor designed to interest more CAS students in pursuing MAs in teacher education. Q 21. Is there validity to the auditors’ perceptions of more liberal allowance (or elective opportunity) in terms of liberal arts and content electives at CAS versus Steinhardt? If so, is this difference a matter of deliberate policy, or has it simply emerged over time as the Morse Academic Plan has evolved? A. Indeed, the Morse Academic Plan (MAP) requirements in the College of Arts and Science have evolved since the curriculum was first implemented. These changes were made to avoid duplication of course content and instructor resources between the MAP and Faculty of Arts and Science departments, while also enabling students to better integrate their general education foundation in MAP with the requirements of their departmental major. CAS students are thus permitted to fulfill some of their MAP requirements by taking departmental courses that have been reviewed and approved by the faculty steering committees for the two central components of the MAP curriculum – the Foundations of Contemporary Culture (FCC) and the Foundations of Scientific Inquiry (FSI). In FSI, the MAP continues to offer courses in Quantitative Reasoning, Natural Science I (physical science) and Natural Science II (life science). However, students can fulfill their Quantitative Reasoning requirement by taking an approved course in statistics offered by the Psychology, Economics, or Politics departments. Similarly, students can fulfill their Natural Science I and II courses by taking approved courses offered by the science departments. In FCC, MAP continues to offer courses in Expressive Culture, but there are approved departmental courses that allow students to fulfill this requirement. The Societies and Social Science requirement have been completely departmentalized to avoid redundancy between MAP and departmental course offerings. It should be stressed, however, that two MAP courses are still required of all students – Texts and Ideas (formerly Conversations of the West) and Cultures in Contexts (formerly World Cultures). It is true that the Steinhardt core liberal arts requirements tend to follow the older MAP scheme and thus incorporate fewer elective opportunities. However, Steinhardt students may take some portion of their liberal arts courses in Steinhardt (see Question 17 above). Ultimately, the Steinhardt faculty decides how MAP is implemented within its majors. One of the benefits of this audit, however, is that it alerts both faculties to differences in the MAP experiences of CAS and Steinhardt students – which again are good matters for the Teacher Education Council to discuss and address as needed.

Faculty Q22. Tracking ratios of tenure-‐track instructors to non-‐tenure-‐track instructors, and also full-‐time instructors to part-‐time instructors, are important means of checking on quality, but the auditors found that these are very difficult to do given current data limitations. Has NYU considered addressing these limitations? A. The auditors expressed frustration with having to check with different department administrators in


101

order to obtain information on the adjunct faculty. The TEAC Coordinators’ subsequent discovery of a faster route (see answer to Q 30) might have assuaged this frustration. Meanwhile, Steinhardt can identify ratios of tenure-‐track to non-‐tenure-‐track instructors through its in-‐house database of faculty appointments in the Office of Faculty Affairs, and even on the website which lists faculty by rank. It is also possible that the new Peoplesoft information system may make merging data from separate databases for reporting and accountability purposes easier than it currently is. Q23. How does NYU monitor the balance in faculty expertise and experience available to students? For example, is this a routine part of advisement and course staffing? A. One virtue of the Internal Audit as it was conducted this time, and of the Shadowing exercise tied to it, is that it made manifest the possibility of such imbalance. Following the recommendation of the auditors to hold internal audits on this same design at least every other year will provide a means of monitoring imbalance and checking on efforts to curtail it. These efforts should include – as this question suggests – making the monitoring of balance/imbalance in faculty expertise and experience a routine part of advisement. There is little evidence that it is now. However, it would be easy to change this, since all teacher education students must be cleared each semester by an advisor before they are allowed to register for courses. One element of this clearance should involve a check on faculty expertise. Meanwhile, department chairs and program area directors should also keep balance in mind as they staff the courses that comprise program requirements. They should ensure as well that all senior and tenure-‐track faculty teach courses in the teacher education program, and teach undergraduates as well as MA students. Where particular program areas seem too weighted toward clinical and/or part-‐time faculty expertise, chairs and program area directors should shuffle requirements as needed – including opening more slots to electives – to ensure that students may also study with tenure-‐track and senior professors. Finally, the Deans should be alert to the possibility that some program areas may suffer imbalance and should take this into account in developing faculty hiring plans. Q24. As the result of the recent addition of an arts-‐focused staff member to the Office of Clinical Studies, the monitoring of quality with regard to the supervision of field placements will be one significant step easier, but are further consolidating steps contemplated – for example, storing duplicate copies of CVs in the Office of Clinical Studies? A. All supervisors’ CVs – whether they are part-‐time or full-‐time faculty – are stored in the department administrators’ files. This makes sense from an efficiency point of view, given the fact that the department administrators process all matters regarding supervisor appointments. Q25. The mentoring and evaluation of full-‐time Steinhardt faculty seems well articulated with standards across departments, but what about part-‐time faculty? A. The part-‐time (adjunct) faculty is very important to the success of the NYU Teacher Education Program (see more on this in answer to Q53 below). However, adjunct faculty members differ considerably from each other, in role and in the curricular niche they fill – ranging, for example, from a notable professional theatre artist or currently practicing high school social studies teacher whose own work environments constitute part of their contribution, to a science education doctoral student or recently retired NYC literacy coach who teach courses on campus. This is one reason that their hiring, mentoring, supervision, and evaluation have always rested, for the most part, at the level of program areas and departments. Another reason is to ensure that their practice is well connected to the practice of the full-‐time faculty, and that their expertise contributes to the program areas’ communities of practice. The exception to this “local” approach is that the work rules for part-‐time faculty at NYU are governed by a collective bargaining contract worked out at the level of the whole university, and


102

overseen for teacher education by the Steinhardt Office of Administration and Finance.

Candidates Q 26. Steinhardt and the Graduate School of Arts and Science have recently scaled up the French program to Chinese and Spanish language studies also, but are the systems of advisement and data sharing within and between the schools sufficient to the challenge? A. The two schools jointly operate a teacher education program in French, and are planning (pending final state approval) a teacher education program in Spanish and TESOL. These programs are in collaboration with NYU Study Abroad sites in Paris and Madrid respectively. There is no comparable program in Chinese, however. The auditors likely confused the French and Spanish/TESOL certification programs with a program for native Chinese speakers who study in a new program jointly operated by Steinhardt and East China Normal University, and who plan to return to China to teach English. The latter does not include New York State teacher certification As for the challenges of launching and operating such joint programs, the French program is instructive in alerting the faculty to predictable start-‐up kinks and also to effective ways of addressing them. The program recruits students throughout the U.S. who spend the first year in Paris (at NYU/Paris), and the second year in New York City (mostly at Steinhardt). Professor Shondel Nero who oversees Steinhardt’s participation recently made an assessment of the experiences of the program’s first two cohorts, involving interviews of students and faculty, and a visit to the Paris Center. She found a number of problems, mostly involving the second year. One was “reverse culture shock” -‐ students getting used to New York after a year in Paris. Another was students’ fear of losing their French fluency gains during their time in New York – especially given the second year’s focus on student teaching and its fairly light contact with the French Department. And third was students’ unfamiliarity with New York City schools and classrooms. She and her colleagues have addressed these problems by ensuring some on-‐site contact with Steinhardt faculty during the Paris year – focused particularly on learning about New York City schooling, and efforts to tweak the New York curriculum and to enhance the role of the French Department during the New York year. These solutions will also, of course, inform the launch of the new Spanish program. Q27. What is NYU’s plan for ensuring that all supervisors will be DRSTOS trained? A. As the Brief indicates, NYU has made great progress since its first TEAC review in scaling up the DRSTOS-‐R assessment. Moving to 100% implementation, however, involves certain complications, as follows. First, there is the resource challenge. Supervisors must be trained, and the added time for training must be compensated. But even if the kind of resources that funded the original scale up (a grant from the Carroll and Milton Petrie Foundation) were still available, initial training is insufficient, as the DRSTOS-‐R data disaggregated by program area reported in the Brief begin to suggest. Ongoing training is necessary to ensure inter-‐rater agreement. This is one of several factors which seem to call for a re-‐conceptualization of the supervisor role in NYU teacher education. Thus supervisors may need to be more integrated with the full-‐time faculty who form the program area communities of practice. All may need to be linked by technologies that permit ongoing, job-‐embedded training in assessing teaching, and in mentoring those assessed in a standards-‐based way. This might be part of a re-‐conceptualization that also expands the supervisors’ contacts with candidates as well as with cooperating teachers, or even merges the roles of supervisor and cooperating teacher. Current experiments with residency-‐based teacher education models – including some at NYU – are taking these paths. Meanwhile, New York State is about to pilot a statewide portfolio assessment of student teachers based (like DRSTOS-‐R) on the Danielson Framework. NYU will participate in the pilot. It is possible that


103

the roll-‐out of this new set of standards and assessment system will make DRSTOS-‐R obsolete. For the moment, however, it makes sense for NYU to make modest investments in DRSTOS-‐R training and re-‐training, and discuss and experiment with substantial modification of the current supervisory role. Q28. How do we ensure that DRSTOS-‐trained supervisors complete and submit DRSTOS documents to CRTL? A. The DRSTOS-‐R training and data collection processes have been designed to ensure and facilitate supervisors’ completion and return of documentation at the end of each semester. During initial DRSTOS-‐R training, it is made clear to participating supervisors that completing the DRSTOS-‐R form is an ongoing expectation with an explicit statement of this expectation as well as discussion regarding how collection of evidence to inform DRSTOS-‐R ratings and assignment of ratings can be integrated into supervisors’ existing mentorship and assessment practice. CRTL sends supervisors a minimum of two reminders per semester – one at mid-‐term and one approximately two weeks before the assigned deadline. Supervisors are contacted using the email addresses provided to the clinical field office and/or any email addresses indicated as preferred. Submission deadlines are timed to coincide with the due dates for final grade submissions to assist the summative rating process. Supervisors have the option of returning digital or hard copies of the DRSTOS-‐R form according to their preference and hard copies are provided by request. In addition, supervisors are sent a confirmation email both when hard copies of the forms are provided and when forms have been received. If forms have not been received by the deadlines, follow-‐up emails are sent to supervisors every 2-‐3 weeks to check on the forms’ status until a month into the following semester. CRTL is in regular contact with program department heads and representatives from the Office of Clinical Studies who are also encouraged to make further reminders in their meetings with supervisors. As needed, CRTL addresses any questions or concerns regarding the DRSTOS-‐R document and its administration via email and individual and/or group meetings. Any supervisors who have not returned forms by the mid-‐term of the following semester are considered non-‐submitters. Return rates since fall 2007 from trained supervisors in the field have been upwards of 80% per semester. In fall 2010, 65 of 70 trained supervisors in the field (92.9%) returned completed forms to CRTL. The 371 forms collected represented 92.1% of student teachers for the semester. Q29. Are the roles of and relationships among advisors, Certification Officer, and the staff member in the Office of Clinical Studies who processes paper work related to certification articulated clearly for the benefit of students? And are these as efficient as possible? A. Progress toward meeting certification requirements is fairly straightforward and articulated to students throughout their program so that they receive this information more than once, oftentimes 3 and 4 times. NYS Certification requires: ·∙ Fingerprint clearance ·∙ Passing scores on the required NYS certification exams (reported directly to NYS) ·∙ Completion of a state-‐approved teacher preparation program. Fingerprinting is required by the NYCDOE before students can student teach, and happens well before graduation. NYCDOE then shares fingerprinting clearance with NYSED so students do not need to be fingerprinted again. During orientations (fast track), fieldwork seminars, student teaching seminars and stand-‐alone certification workshops, the Certification Officer provides and reviews with students both the step-‐by-‐step online application instructions for NYS certification, and an information packet outlining the


104

additional requirements for certification (exams, fingerprint clearance, etc.) in addition to degree requirements. Students are advised which exams they need to complete to satisfy NYS certification requirements, and warned that without the exams, NYSED will not certify them. The Certification Officer cannot monitor whether candidates register to take the exams until a score (pass or fail) appears in the TEACH system (NYSED's online database). Not all students complete the NYS certification exams. Some do not because they plan to move out of state upon graduation. Some undergraduates decide that they will complete their MA before they start teaching, and thus complete the exams while in graduate school. Some students complete the exams during the summer after they've graduated. CRTL monitors exam scores via the TEACH system as part of its annual reporting to the faculty and TEAC. Steinhardt does not recommend students for certification (nor complete out of state verification forms) until the degree is completed. The staff member in the Office of Clinical Studies (referred to in the question) is also the Certification Liaison for Teaching and Learning, and recommends only the graduates of Teaching and Learning electronically on the NYSED TEACH system. The Certification Officer recommends all other program completers electronically. NYSED provides access to TEACH to three individuals from each university. Progress in terms of academic requirements for certification is the same as progress toward degree completion which is monitored by the student’s advisor. Academic progress in terms of GPA requirement is monitored by Steinhardt Registration Services. Q 30. How does NYU track and confirm admitted candidates’ efforts to make up deficiencies in admission pre-‐requisites? A. Teacher education MA applicants may be admitted with a deficiency of no more than 6 credits in their content specialization. The Office of Graduate Admissions provides a statement to each student who is missing credits that specifies how many credits and in what areas of study the student is deficient. Those credits must be taken at NYU in excess of degree requirements. A copy of the statement is also sent to the program adviser who meets with every student at least once a semester and reviews their progress toward degree completion, including missing pre-‐requisites. Q 31. Does NYU have plans to develop a new and online or hybrid mentoring program, following the lead of other teacher education institutions? A. A number of recent high-‐profile reform initiatives and policy papers on teacher education have suggested that the problem of low retention of early career teachers – an expensive problem for urban districts in particular – could be assuaged by post-‐certification mentoring programs. Indeed, NYU has for the past 5 years had a program in place, called the New Teacher Support Network, which supported recent graduates teaching in New York City by means of a series of evening seminar sessions on topics of great interest to them – for example, teaching English language learners and classroom management. These were held in partnership schools, and drew equally on school and NYU expertise. The program was funded by grants from the Wachovia Foundation and the Petrie Foundation. But the end of the funding cycle, combined with a NYC job freeze ended this initiative. Online mentoring programs, like the one that Teach for America has created and recently presented at an NYU meeting, can be very expensive. TFA’s program cost millions of dollars to build, and requires a dedicated staff to maintain. Neither NYU nor Steinhardt have the funds to launch such an elaborate support structure at this time. At this point, the teacher education faculty should consider how it might deploy existing resources at a more modest scale and to the same end. Online is clearly the way to go if for no other reason than NYU graduates cannot depend for jobs on the local market alone. The Steinhardt administration could facilitate and assist this effort – for example, by pioneering ways to use 1-‐ 3 credits of tuition funding on


105

post-‐graduation support, and by exploring the use of blended learning platforms (those that combine short-‐term coursework and resource libraries with social networking). Q32. The overall process of transitioning candidates from student status to certified teacher status is divided among various offices and roles: the Wasserman Center, the Certification Officer, departmental advisors who monitor progress, a staff member in the Office of Clinical Studies who handles final certification paperwork, a departmental listserv that posts job vacancies, and so on. Has NYU considered the possibility of redesigning for greater integration and efficiency? A. The steps involved in obtaining certification are listed above in response to another question. While there are always ways to improve efficiency, the actual certification process is more efficient than ever. Students now apply online for certification, exams scores are reported electronically to the NYSED and appear in their TEACH account, fingerprinting is done prior to student teaching by the NYCDOE and clearance forwarded electronically to the NYSED, and then institutions recommend electronically Although the Wasserman Center and the job vacancy listserv play no role in the certification process, they are, of course, useful in the employment of our students as teachers. Q33. Teacher education at NYU is highly prescriptive in terms of course requirements, with few or no non-‐restricted electives depending on certification patterns. Students with special needs and interests therefore require significant advisement involvement. Does everyone who seeks flexibility get this kind of advisement? Would broader elective opportunities better serve student interests and needs? A. Indeed, most of the program areas of NYU teacher education offer few and often highly restricted elective opportunities. Some of these constraints are products of state mandates and of how previous faculty groups have interpreted these mandates. Recent perceptions on the part of the faculty and Deans of the need to reduce overall credit requirements in teacher education may further reduce elective opportunities. However, there are countervailing trends. One is an effort – notable within the Department of Teaching & Learning -‐ to create greater economies in terms of faculty deployment by opening up courses previously devised for only one program area to several areas. Another is associated with a schoolwide effort to create courses that will attract students from other NYU schools to the study of education. These courses are likely to work best if they are constructed to appeal to Steinhardt students too, and if Steinhardt students can satisfy particular certification requirements by taking them. Finally, there is the effort to afford all NYU students – including those in teacher education – opportunities to study at one or more NYU study abroad sites. Such study is easier in a curricular environment that does not require that program areas be precisely duplicated site to site. Q34. Is there a consistent system across certification patterns, levels, and departments in how requested substitutions for program requirements are handled, negotiated, approved, recorded, and passed along to the Registrar? A. The policy is very clear within Steinhardt and NYU as a whole -‐ namely that exceptions to program requirements are proposed by program area directors, approved by the Dean of Faculty & Academic Affairs, recorded with the Office of Graduate Studies or the Steinhardt undergraduate advisement office, and ultimately decided upon by the Registrar. Q35. Given the emphasis in current policy advocacy and in NYU’s TEAC claims on teacher content knowledge, should grades below C-‐ count toward content requirements? A. The finding that provoked this question will lead to faculty deliberation this fall. Indeed, in its


106

guidelines regarding transcript review (of teacher candidates who have not graduated from a New York State approved teacher education program) NYSED says that undergraduate content course grades below C may not count toward meeting content core certification requirements. See http://www.highered.nysed.gov/tcert/ag.html. For graduate students, no content core course below B-‐ may count – again for the purposes of transcript review. The state defines “content core” as “coursework that instructs candidates in the specific subject matter of the certificate title sought (e.g. Mathematics, Biology, Spanish, etc.).” Because the NYU teacher education program is an approved program, however, it is exempt from this policy, and the advisor was right in telling the Auditor that the Steinhardt policy requiring at least a 2.5 overall GPA for undergraduate and graduate students to progress to student teaching is operative here. TEWG recommends that the faculty amend this policy to reflect the higher transcript review guidelines.

Resources Q36. Are courses in teacher education expected to list relevant resources on the syllabus, and to tag these for easy reference in the library and online? A. Most faculty at NYU would regard this as good teaching practice at the university level. However, there is no explicit Steinhardt policy requiring this. One virtue of going online with all syllabi – as the College of Arts and Science recently did – is to make examples of good practice more available for emulation. See Question 4. Q37. What policies or review mechanisms exist at Steinhardt to ensure that undergraduates get a fair share of the attention of full-‐time and tenure-‐track as well as tenured faculty? A. What the auditors perceived in this regard deserves a fuller-‐scale study. And, if indeed, particular sub-‐groups of students – by virtue of their status as undergraduates, their program areas, or other factors – are disproportionately taught by part-‐time faculty, then the faculty must take steps to address the problem. Q38. What balance point does Steinhardt aspire to in terms of the proportion of full-‐time and part-‐time faculty teaching in its teacher education program? A. NYU has taken a number of steps to bring the NYU percentage of full-‐time faculty teaching Teacher Education program students from 42% reported in the 2005 TEAC Brief, to 48.4% in 2011. These steps have included converting a number of highly regarded adjunct or visiting instructors into full-‐time clinical faculty or master teachers (notably David Montgomery in Educational Theatre, Debbie Damast in Dance, and Rosa Pietanza in core studies). Future conversions are also likely, as are reductions in the overall number of courses required, and consolidation of courses. In the end, NYU aspires to reach 60% full-‐time faculty with an overall mix of faculty that includes tenure-‐track scholars, clinical faculty with deep practical knowledge, and master teachers and adjunct instructors with great currency in practice.


107

REFERENCES Jeffery, J.V. & Polleck, J. V. (2010). Reciprocity through co-‐instructed site-‐based courses: Perceived benefit and

challenge overlap in an urban school-‐university partnership. Teacher Education Quarterly, 37 (3), 81-‐99. Levine, A. (2006). Educating school teachers. Washington, D.C.: The Education Schools Project

[http://www.edschools.org/pdf/Educating_Teachers_Report.pdf]. McDonald, J. P., Mohr, N., Dichter, A. & McDonald, E. C. (2007).The Power of protocols. NY: Teachers College

Press. Meier, J. & Crowe, E. (2009).Evaluation of the Partnership for Teacher Excellence for the Carroll and Milton

Petrie Foundation. New York: Arete Consulting. Poliakoff, A.R., Dailey, C.R. & White, R. (2011).Pursuing excellence in teacher preparation: Evidence of

institutional change from TNE Learning Network universities. Washington, D.C.: Academy of Educational Development.


108

Appendix B

Evidence of Institutional Capacity for Program Quality

INTRODUCTION

The NYU Teacher Education Program is situated within one of the world’s largest private institutions of higher education: 18 schools and colleges, enrolling some 50,000 students. It occupies 5 million square feet of interior space at five centers in Manhattan, one in Brooklyn (the NYU Polytechnic Institute), and campuses throughout Europe as well as in the Middle East, Africa, Asia, South America, and (recently announced) Australia. NYU has 3100 full-‐time faculty, including some of the most distinguished in the world, and a curriculum that combines more than 2500 courses with one of the most extensive set of professional internships in the world. In 2006, NYU was classified by the Carnegie Foundation for the Advancement of Teaching as one of 76 “institutions of community engagement,” in recognition of its extensive involvement in community-‐based teaching, learning, and research. In 2009, NYU’s research funding exceeded $270 million. From 2004 to 2008, the University received more income from the licensing of technologies developed at NYU than any other U.S. university, and in the past 20 years more than 55 start-‐ups have been formed around NYU discoveries and ideas. NYU’s eight libraries hold over 4.5 million volumes. The University’s student support services have won multiple awards: its career counseling service won excellence awards in 2007 and 2010 from the National Association of Colleges and Employers; its Student Health Center won a 2010 American College Health Association award for innovative practices; and in 2009 and 2010, NYU’s student services were awarded six Gold Excellence and two Silver Excellence awards from the National Association of Student Personnel Administrators.

NYU teacher education is situated within the Steinhardt School of Culture, Education and Human Development, an undergraduate and graduate professional school that is the home of the third largest full-‐time faculty at NYU (after Arts and Science, and Medicine). Steinhardt is currently ranked 15th in the 2011 U. S. News and World Report rankings of the country's top graduate schools of education. In addition to the 61-‐member Teacher Education Faculty group, the 270-‐member full-‐time Steinhardt faculty includes groups in music and the arts (with whom music and art education faculty are integrated), as well as in media and communications, leadership and technology, applied psychology, humanities and social sciences in the professions, and health (nutrition, speech, physical therapy, and occupational therapy). Steinhardt’s mission is to advance knowledge, creativity, and innovation within all these areas and wherever possible at their intersection too, and to produce professionals conscious of the connections among professional practice, cultures, and urban communities worldwide. The school enrolls nearly 6200 students – of whom 2500 are undergraduates and 3700 Masters, advanced certificate, and doctoral students. Beyond diversity in their professional interests, Steinhardt students are diverse geographically, ethnically, and culturally. The undergraduates are 99% full-‐time and predominantly residential. About half of them begin as freshmen, admitted as part of NYU’s university-‐wide undergraduate admissions process, and primarily spend their first two years studying the liberal arts beside undergraduates from the College of Arts and Science and NYU’s other undergraduate schools. The other half of the undergraduates come to Steinhardt at a later point -‐ when their professional interests begin to gel -‐ and either as internal transfers (within NYU) or external ones (from other four-‐year colleges or community colleges). Altogether the Steinhardt undergraduates are 71% female, 3% African-‐American, 9% Latina/o, 15% Asian-‐American, and 4% international. The graduate students are predominantly Masters-‐level, 60% full-‐time, 79% female, 8% African-‐American, 11% Latina/o, 9% Asian American, and 16% international. Steinhardt has an extensive student services department that includes both an undergraduate and a graduate student organization, but Steinhardt students also have easy access to an even larger array of student services and activities maintained by the University centrally – including medical and counseling services, disabilities services, recreational services, and career services.

The Teacher Education Program enrolls approximately 20% of Steinhardt’s undergraduate students and


109

15% of its graduate students, within program areas that run from early childhood and childhood through secondary foci in nearly every school subject. Teacher education has the third largest student enrollment at Steinhardt – after music and media/communications. However, the program is among the most inter-‐connected of majors at NYU, working closely with the Faculty of Arts and Science, as well as the Steinhardt music and arts faculties, humanities and social science faculty, applied psychology faculty, and technology faculty, in such areas as undergraduates’ content and liberal arts content knowledge, and both undergraduates’ and graduates’ knowledge of the learning sciences and of the contexts and tools of teaching. The NYU Teacher Education Council and Teacher Education Working Group, with members from Arts and Science as well as these multiple Steinhardt faculty groups, oversee the ongoing development of the Teacher Education Program.

The NYU Teacher Education Program places its candidates early and at multiple points in scaffolded field

experiences – including NYU courses and seminars that meet in schools, observation assignments in schools and other community-‐based learning centers, tutoring and small-‐group teaching assignments, and student teaching. It has strong ties to New York City schooling. These include ties with the central New York City Department of Education (NYCDOE), with early childhood centers operated by multiple agencies and institutions, and with NYCDOE elementary and secondary schools in Manhattan, the Bronx, and Brooklyn. Occasionally, the program makes placements outside New York City too – typically for foreign language teaching (for example, in French or Chinese), or because of students’ special needs. The program also has a special relationship with a set of 22 NYC partnership schools – a relationship that has significantly expanded the program’s resources, including expertise, space, and learning opportunities for students and faculty. The purpose of the partnership is to strengthen teacher education and contribute to school improvement and learning gains for K-‐12 students by means of deeper and wider ties between school and university. The partnership schools and NYU have signed a memorandum of understanding which articulates a commitment to work in the interest of each other’s students, by means, for example, of fieldwork that immerses teacher candidates in the teaching and learning life of the partner school, appointments of selected members of the schools’ faculties to the NYU adjunct faculty, and opportunities for the schools’ teachers and students to take advantage of learning opportunities available at the university. RESOURCE CAPACITY FOR QUALITY: DOCUMENTATION AND COMPARATIVE ANALYSIS

Table B.1 (Table 5 data) compares the Steinhardt Teacher Education Program to NYU overall and to other NYU schools and departments with regard to dimensions of resource capacity for quality. Table B.2 (Table 6 data) below provides documentation regarding NYU’s compliance with the TEAC requirements regarding resource capacity for quality.


110

TABLE B.1 [Table 5 data]: Capacity for Quality: A comparison of program and institutional statistics Capacity Dimension Program Statistics Institutional Statistics Difference/ Analysis 3.1.1 Curriculum (number of credits)

Undergraduate Teacher Education To be eligible for a baccalaureate degree, students complete a minimum of 128 credits within 10 years of matriculation. The Teacher Education Program varies slightly among program areas (for example, 128 in science, math, and foreign language, but 129 in English, and 133 in educational theatre). Joint certification programs are still higher (for example, 137 in childhood/special education). http://steinhardt.nyu.edu/teacher_certification/#music1

Undergraduate Studies at NYU A minimum of 128 credits are required for the NYU baccalaureate degree, with some requiring in excess of this

Comparison/analysis No difference: The minimum number of credits required for the baccalaureate degree (either BA or BS) is 128, though some students take more, and dual majors may require more. http://cas.nyu.edu/page/ug.MajorsMinors.html

Graduate Teacher Education Master’s Degree requirements for initial teacher certification range from a minimum of 36 credits to a maximum of 53 credits for the Dual Certification Program in Ed Theatre/English. Graduate students in Masters’ programs need to maintain a 2.5 GPA.

Graduate Studies: GSAS, Wagner, Steinhardt Media, Culture & Communications To be awarded a Master of Arts and Master of Science Degree in the NYU Graduate School of Arts and Science, a student will complete at least 32 points of graduate credit (at least 24 in residence at the Graduate School, 16 points in one department or program) and have a cumulative GPA of 3.0 or better. http://gsas.nyu.edu/object/bulletin0911.grad.admission

Comparison/analysis There is variability in the number of credits required for a Master’s degree within the Teacher Education Program (by area), and across NYU programs and areas generally


111

3.1.2 Faculty (percentages at ranks; workload; diversity)

Full-‐time Teacher Education Faculty percentage at ranks. Table B.3 below lists Steinhardt faculty in terms of percentages at ranks.

Full-‐time University Faculty percentage at ranks Table B.4 below compares the Steinhardt faculty to the overall university faculty in terms of percentages at ranks.

Comparison/Difference The Steinhardt percentage of faculty at full professor is lower (by 3.9%) than at the university as a whole, and thus those at associate and assistant levels are proportionally higher (3.4% and 2.4% respectively). In the clinical ranks, Teacher Education makes more use of the Teacher/Master Teacher rank, and almost no use of the Instructor rank. Otherwise the differences are minimal.

Adjunct Teacher Education Faculty In the 2010-‐2011 academic year, 48% of teacher education program courses (excluding content area courses and liberal arts courses for undergraduates) were taught by full-‐time faculty members.

Adjunct other NYU Faculty In the 2010-‐2011 academic year, 32% of courses in the Steinhardt Department of Media, Culture and Communication were taught by full time faculty.

Comparison/difference There is wide variability across NYU schools in terms of the use of adjunct faculty. There are some generalities, however. One is that when graduate students teach, they are appointed as adjunct faculty. Another is that the professional schools make great use of the field’s practitioners as adjunct faculty. This is true in both teacher education and media communication at Steinhardt .


112

Faculty Workload in Teacher Education. A Steinhardt task force has recently studied issues related to faculty workload, and will report recommendations to the faculty. The task force was, in part, born out of a need to capture statistical data on teaching loads—which we currently do not have. The findings of the task force may result in adjustments to the current norm which generally assigns research-‐active tenure-‐track faculty a teaching load of two courses per semester, and clinical/master teacher faculty a teaching load of three courses per semester – with adjustments downward where research buy outs are involved, and equivalencies provided for student teaching supervision and administrative assignments (such as program area chair).

Faculty workload in the Faculty of Arts and Science There is some variation among departments and between the sciences and the humanities (given administrative assignments and the frequency of course buy-‐outs for major research projects in the sciences). In general, however, members of the Faculty of Arts and Science teach two courses per semester.

Comparison/Difference The norms are the same, and variation from the norms occurs for the same reason (i.e., research or other project buyouts, and administrative assignments).

Diversity of the Teacher Education Faculty. Table B.5 below provides statistics on teacher education faculty by gender and ethnicity.

Diversity of other NYU faculty Table B.5 below compares the Steinhardt faculty to the overall university faculty by gender and ethnicity.

Comparison/Difference In terms of diversity of faculty, Steinhardt’s percentage of women faculty is 59% versus 41% at NYU overall. Moreover its percentage of Black and Latino/Latina faculty is 12% versus 5% at NYU overall.

3.1.3 Facilities (space & equipment provided)

Teacher Education Program The Teacher Education Program shares equally in many facility resources with other NYU units. For example, note the references above concerning classroom assignments, technology support, and library resources.

Other Steinhardt: Media, Culture & Communication (MCC) The Dept. of Teaching & Learning (T&L) (educating most of the teacher education students) generates 9100 student credits per semester – only the third highest at Steinhardt, but it has the largest

Comparison/Difference The comparison is mixed, but suggests a functional basis with no systematic bias. The Teacher Education program includes students from Music and Art (which have the highest OTPS budgets and space allocations at


113

full-‐time faculty, and at 12,426 sq. ft., it also occupies more space, and has a higher OTPS budget than MCC, the second highest enrollment department.

Steinhardt). However, most teacher education students are in the Dept. of Teaching & Learning which is first in faculty size and third in both OTPS and space).

3.1.4 Fiscal and administrative (support dollars/faculty member)

Fiscal support per faculty in teacher education Table B.6 below provides a Comparison of Faculty Salary (Mean) for Steinhardt and the University, 2008-‐2010. Individual Development Account (IDA) funds are also available to all Steinhardt faculty to support professional development: http://steinhardt.nyu.edu/adminfinance/howdoi-‐2010-‐02-‐22-‐IDA\

Fiscal support for faculty university-‐wide See Table B.6.

Comparison/difference There are significant differences: for Professors, a gap of roughly $43,000; for Associate Professors, $11,000; and for Assistant Professors, $22,000

OTPS The OTPS allocation to Steinhardt Departments for fall 2011 is $ 2,926,335. Of this amount, Teacher Education receives $463.503 or 15.8% ( based on fall 2010 enrollment points of 1111 Teacher Education majors)

Other Steinhardt department: MCC By comparison, of the total $2,926,335 OTPS allocation for departments, MCC received $204,910 for fall 2011 or 7% of the budget based on fall enrollment points of 914 MCC majors.

Comparison/difference Within Steinhardt, the teacher education program is among the most highly funded programs. For example, the Dept. of Teaching & Learning, which has the bulk of teacher education students but generates fewer credits than either the Media or Music depts., has many more full-‐time faculty.

The Steinhardt financial report. A copy is available in the Dean’s Office and in the TEAC Coordinator’s Office. Of the $212,155,000 Steinhardt 2009-‐2010 budget, 52.7 million (26%) was allocated for Instructional salaries and 11 million

Overall NYU Financial Report. The report for 2009-‐2010 is available at: http://www.nyu.edu/financial.services/cdv/pdf/CFS_2010.pdf See also: http://www.nyu.edu/administrativ

Comparison/Difference NYU schools and programs within schools are funded on the basis of annual budgets negotiated between Deans and Provost at the University level and between Deans and Department Chairs at the


114

(5%) for OTPS e.services/ http://www.nyu.edu/budget2010/budget/ http://www.nyu.edu/about/leadership-‐university-‐administration/office-‐of-‐the-‐president/office-‐of-‐the-‐executivevicepresident/finance-‐and-‐budget/budget-‐and-‐planning.html

school level. Funding varies in terms of University and School priorities. It should be noted in this regard that the University has recently added $45 million in debt to help Steinhardt finance the capital campaign that will put all of Teaching and Learning and all teacher education program areas in other departments into better, more usable, more contiguous, and technologically sophisticated space. The total capital plan is about $110 million.

3.1.5 Student support services (equal access to services)

Teacher education student support services All Teacher Education majors are assigned advisors. Student complaints are addressed following the policies and procedure indicated on the website: http://steinhardt.nyu.edu/policies/procedures All teacher certification program applicants are considered for financial aid and school-‐based scholarships. Decisions are based on merit of the application as well as financial need as determined by the FAFSA. Additionally teacher education students are eligible to receive scholarships, which are exclusively for education students, such as the Jewish

University-‐wide Student Support Services NYU Office of Student Affairs: provides students support services as noted at: http://www.nyu.edu/about/leadership-‐university-‐administration/office-‐of-‐the-‐president/office-‐of-‐the-‐provost/university-‐life/office-‐of-‐studentaffairs.html and advisement: http://www.nyu.edu/advisement/your.advisor/

Comparison/difference Student Support services provided to Teacher Education majors are parallel to services provided by other NYU Schools. Teacher Education majors have access to all University-‐wide Services.


115

Foundation for Education of Women, The Noyce Foundation, The Gateway Project, and Math for America. For 2009 and 2010, on average, 27% of entering students (schoolwide) received a school-‐based scholarship. The 2009-‐2010 financial aid provided to Steinhardt students was $26 million -‐ (13%) of the overall Steinhardt budget, Excluding externally funded scholarships, the proportion expended on teacher education students was proportional to their share of credits. Individual scholarship amounts covered between 20% and 40% of tuition costs.

3.16 Student feedback (course evaluation, mean number of complaints)

Teacher Education Program course evaluations Students routinely complete course reaction forms at the end of each semester. CRTL currently provides analysis of these evaluation forms and the results are shared with department chairs and program directors as well as with individual faculty members. Steinhardt is currently in the process of moving to an online course faculty evaluation system. One department piloted the online system in spring 2011.

Other NYU The NYU College of Arts and Science provides a Course Evaluation Guide compiled from student surveys dating from the year 2000 to present. Results for FAS are made available online http://www.nyu.edu/cas/ceg/ In Stern, all students must fill out an online Course Faculty Evaluation (CFE) for each course for which they are registered. The responses to all online evaluations are completely anonymous and confidential. Students may fill out the CFE online through the following link: https://ais.stern.nyu.edu

Comparison/analysis Steinhardt is currently moving to online course evaluations for all Steinhardt programs-‐ consistent with NYU FAS, Wagner, and Gallatin, and will be posting results of the evaluation online beginning with the spring 2012 semester


116

NYU Wagner’s website contains the faculty evaluation by students for courses offered at Wagner during each semester. This is a password protected site. Only Wagner students, faculty and administration with valid NYU IDs have access to this section. http://wagner.nyu.edu/cgi-‐bin/evaluation03NEWe.cgi?start=0&end=0&table=evaluation

Teacher Education student complaints. Student complaints are not systematically tracked within Steinhardt or across NYU. However, the Steinhardt School’s Student Complaint Procedure is published in the Steinhardt Student Handbook as well as in departmental materials. https://steinhardt.nyu.edu/policies/procedures#Student%20Complaint%20Procedure Student government also provides a platform for discussing student issues and concerns and referring students to appropriate sources for resolution. All departments elect representatives to both the undergraduate and graduate student governance bodies.

Overall NYU student complaints Student complaints are not systematically tracked within across NYU.

Comparison/difference The Associate Dean for Student Affairs reports that complaints from teacher education students have not been disproportionate to those of Steinhardt students overall.


117

TABLE B.2 (TABLE 6 data): References to institutional documents for each requirement TEAC requirements for quality control of capacity (3.2)

Documentation

3.2.1 Curriculum Credit hours required in the subject matter are tantamount to an academic major. Credit hours required in pedagogical subjects are at least tantamount to an academic minor.

Teacher Education programs are approved by NYSED and adhere to certification regulations which dovetail with this requirement: http://www.highered.nysed.gov/tcert/regulations.html http://www.highered.nysed.gov/tcert/part80-‐3.html Teacher certification program requirements at the baccalaureate and master’s level exceed this requirement. They are included on the Steinhardt website as well as in Appendix D. A list of state registered teacher education curricula leading to initial and professional teacher certification is included in Table 1.1 on page 5 of the Inquiry Brief. Generally, academic minors at the College of Arts and Science (CAS) require 4 courses of 4 credits each (16 credits total). Education minors at Steinhardt for CAS or for students from other NYU schools do not lead to certification except in combination with a reduced-‐credit Masters degree. Credit requirements for academic minors at both Steinhardt and the College of Arts and Science (CAS) are listed respectively at the following websites: http://steinhardt.nyu.edu/teachlearn/minors#why http://cas.nyu.edu/object/ug.academicprograms.crossschoolminors

3.2.2 Faculty Majority of faculty have a terminal degree (major or minor) in the areas of course subjects they teach.

The credentials of the teacher education faculty amply exceed this requirement (see Appendix C.1). Copies of Teacher Education faculty CVs have been collected by the TEAC Coordinators and are available in their files.

The Teacher Education talent pool is enriched by adjunct faculty members (Appendix C.2) and by adjunct supervisors of student teachers (Appendix C.3) available at the Steinhardt password protected website: (www.Steinhardt/secure/TEAC).

Faculty guidelines on recruitment, personnel performance reviews, appointment and professional development are available via the following Steinhardt websites. http://steinhardt.nyu.edu/steinhardt/faculty_affairs/pdfs/faculty_search_guidelines.pdf http://steinhardt.nyu.edu/faculty_affairs/personnel_reviews http://steinhardt.nyu.edu/scmsAdmin/media/users/cb5/Clinical_Guidelines/Clinical_Faculty_Appointment_


118

Guidelines_Final_12-‐7-‐10.pdf http://steinhardt.nyu.edu/faculty_affairs/professional_development_fund_request http://steinhardt.nyu.edu/faculty_affairs/personnel_reviews#annual_reviews Table B.3 in Appendix B shows the breakdown of full-‐time faculty by rank for Steinhardt overall and for the teacher education faculty.

3.2.3 Facilities Facilities are appropriate and adequate.

NYU is currently embarked on a multi-‐million dollar capital project to upgrade existing instructional, laboratory, research, and faculty office space. Recent specific projects include the complete renovation of the 6th floor of the Education Building (home of the Teacher Education Program areas of Music, Educational Theatre, and Dance) to house new performance and recording space, and a campus-‐wide renovation of general purpose classroom space to afford greater access to technologies for teaching. Forthcoming projects (for which specific plans are available) include the complete renovation of the East Building, home of the Department of Teaching and Learning, which will integrate the Department’s currently dispersed faculty office and meeting spaces, and include a state-‐of-‐the-‐art science education laboratory. http://www.nyu.edu/about/leadership-‐university-‐administration/office-‐of-‐the-‐president/office-‐of-‐the-‐provost/redirect/academic-‐facilities.html

Classrooms are assigned by the university based on the number of students registered in the class. Web pages describing classrooms are available at:

http://www.nyu.edu/campusmedia/classrooms/room/ Media is requested at the same website. Most Steinhardt full-‐time faculty have private offices. The exception is Master Teachers who may share an office since they spend the bulk of their time in the field. Access to space and resources for adjunct faculty and students are provided by the department.

Information regarding NYU technology and library systems, drawn from http://steinhardt.nyu.edu/academics/affairs/faq/, is as follows:

Technology: New York University provides networked PCs and Macs with Internet access and electronic mail capability to all faculty and staff. A computer technical help desk and full-‐time network administrators are available for assistance and troubleshooting through Information Technology Services. The University also has six computer labs and special resources for media production, arts technology, and science and humanities computing. ITS provides particularly strong resources for social science and statistics programs


119

that include lectures, workshops, and expert consultations. Also, NYU’s wireless network continues to expand into new areas of the campus.

Library: New York University maintains eight libraries containing over 4.5 million volumes, five million microforms, 500,000 government documents, 80,000 sound and video recordings and a wide range of electronic resources. The main library, the Elmer Holmes Bobst Library and Study Center, is one of the largest open stack research libraries in the country. The main collection continues to grow by more than 140,000 volumes a year. The University also has six specialized libraries that contain over 168,000 volumes, 2,000 periodicals, computer software and audiovisuals. The NYU Television Center, which also acts as a resource, manages satellite downlinks for instructional and educational programming, and provides on-‐campus teleconferencing. Library users have access to “the Arch,” the NYU library’s database of electronic resources as well a virtual reference desk, which allows users to email questions and receive email responses within 24-‐hours or, for more immediate information, to “chat” online with a reference librarian. To support scholarship, the library provides free bibliographic software (RefWorks, ProCite, and End Note). In addition, the library subscribes to various online educational resources such as Education Full-‐Text, Education Index, and ERIC. The NYU library offers full off-‐ and on-‐campus access to the library's subscription databases, e-‐journals and e-‐books. Access also includes access to partner library collections, as follows: http://www.oclc.org/research/activities/shares/partners/default.htm

3.2.4 Fiscal and Administrative The institution housing the teacher education program is financially healthy. Program administrators are qualified for their positions Resources are adequate to administer the program

The overall NYU financial report is available at http://www.nyu.edu/budget2010/ The Steinhardt Office of Administration & Finance, headed by Associate Dean Robert Knight, monitors the school’s financial and administrative resources. The Steinhardt budget is available in the Office of Administration and Finance and also at the TEAC Coordinators’ office.

Resumes showing qualifications of program administrators are available in the TEAC coordinator’s office. They are evaluated annually as are those for faculty and staff.

Each year, departments assess needs and make budget requests. Department chairs meet with the administration to negotiate the departmental budget for the next academic year. Space and budgets are allocated by department. Each year, the School Planning Committee of the Faculty Senate seeks faculty input on how Steinhardt should allocate its budget. Faculty priorities for the School's budget play a crucial role in Steinhardt's deliberations with University administrators about the allocation of School resources.. In 2010-‐2011, they identified faculty recruitment, salary increases, and doctoral support as the top three concerns.

The School Planning Committee meets every month and undergraduate and graduate student government


120

officers are on the committee. Survey results are presented at Steinhardt Faculty Meetings. Copies of the results are available in the TEAC coordinators’ files.

3.2.5 Student Support Adequate student support services are available.

The Office of the Associate Dean for Student Affairs provides a broad array of academic and student services, programs, activities and opportunities to help students be informed, get connected, and access services. See http://steinhardt.nyu.edu/studentaffairs/dean

The Steinhardt Student Affairs website newsletters entitled “student matters” provide up-‐to-‐date information: http://blogs.nyu.edu/blogs/ejf9434/nyusteinhardtstudentaffairs/

Upon initial registration, all Teacher Education students at NYU are assigned advisors – including staff and faculty. Advisors introduce students to their program area, offer course selection advice, and clear students for registration. Services for students are available at NYU Wellness Center http://www.nyu.edu/999/ and at the Student Health Center http://www.nyu.edu/shc/ The Steinhardt website includes a Teacher Certification timetable : providing information on New York State Certification and New York City Public Schools Employment: http://steinhardt.nyu.edu/teacher_certification/timetable Coordinated by the Office of Clinical Studies, the Apprentice System is a comprehensive online platform designed to support and document students’ fieldwork experiences. The site includes the handbook for student teachers/ guidelines for field supervisors, a directory of student teaching sites, and links to teacher education resources: http://steinhardt.nyu.edu/apprentice/default/resources Post graduation-‐ career advisement is available via: http://steinhardt.nyu.edu/portal/current_students and the NYU Wasserman center for career development http://www.nyu.edu/careerdevelopment/ and at: http://steinhardt.nyu.edu/teachlearn/job_strategies The services provided at the Office of Counseling and Student Support Services are listed at: http://steinhardt.nyu.edu/counseling/


121

Students complete the program at reasonable rates.

Support for students with disabilities is provided at the NYU Moses Center: http://www.nyu.edu/life/safety-‐health-‐andwellness/students-‐with-‐disabilities.html The retention rate for NYU Steinhardt undergraduate students has steadily increased from 83.6 % in 1990 to 91.3% in 2008. This mirrors the overall NYU retention rate which increased from 85.5% in 1990 to 92.4% in 2008.The graduation rate for Steinhardt MA candidates that registered in fall 2005 is 87.4%. The retention rate: 91%. For Certification MA students, the graduation rate is 92.9%

3.2.6 Policies An academic calendar is published. A grading policy is published and accurate. There is a procedure in place for students’ complaints to be evaluated. The transfer of credit policy is published and is accurate.

NYU Calendars are published at: http://www.nyu.edu/registrar/calendars/ Steinhardt Academic calendars are available via: http://steinhardt.nyu.edu/registration/calendars The Teacher Education Program adheres to the academic standards listed at http://steinhardt.nyu.edu/registration/standards Steinhardt grading policies are available at: http://steinhardt.nyu.edu/registration/standards#grading_policies The overall NYU grading policy is available at: http://www.nyu.edu/registrar/transcripts-‐certification/grades-‐information.html Procedures for evaluating students’ complaints are listed at: http://steinhardt.nyu.edu/policies/procedures#Student%20Complaint%20Procedure Transfer of credit policy is posted per undergraduates and graduate students. Undergraduate: http://steinhardt.nyu.edu/undergraduate_transfer/transfer_credit Graduate: http://steinhardt.nyu.edu/advisement/masters/transfer_credit


122

TABLE B.3 Steinhardt Full-Time Teacher Education Faculty by Rank, Fall 2010- Spring 2011

Rank Steinhardt

Overall

Number (%)

Steinhardt Teacher

Education Number (%)

Steinhardt Teacher Education Related

Program Faculty Number (%)

Steinhardt Other Faculty

Number (%)

Professor 70 (26.7) 11 (18) 46(37.4) 13(16.7)

Associate Professor 60 (22.9) 13 (21.3) 31(25.2) 16(20.5)

Assistant Professor 48 (18.3) 10 (16.4) 23(18.7) 15 (19.2)

Clinical Professor 6 (2.3) 2 (3.3) 2(1.6) 2(2.6)

Clinical Associate Professor 18 (6.9) 4 (6.6) 5(4.1) 9(11.5)

Clinical Assistant Professor 17 (6.5) 5 (8.2) 9(7.3) 3 (3.8)

Teacher or Master Teacher 20 (7.6) 12 (19.7) 2(1.6) 6 (7.7)

Music Associate Professor 4 (1.5) 0 0 4 5.1)

Music Assistant Professor 8 (3.1) 0 0 8 (10.3)

Visiting Associate Professor 2 (0.8) 1 (1.6) 1(0.8) 0

Visiting Assistant Professor 7 (2.7) 3 (4.9) 2(1.6) 2(2.6)

Other(Assistant Professor/Faculty Fellow 2 (0.8) 0 2(1.6) 0 TOTAL 262 61 123 78


123

TABLE B.4

Comparison of Full-time Steinhardt and University Faculty by Rank, 2009-2011

Steinhardt University Title 2008-2009 2009-2010 2010-2011 2008-2009 2009-2010 2010-2011

Number (%) Number (%) Number (%)

Number (%) Number (%) Number (%)

Professor 74 (27.3) 66 (25.5) 70 (26.7) 721 (31.2) 733 (30.8) 750 (30.5) Associate Professor 63 (23.3) 65 (25.1) 70 (26.7) 460 (19.9) 476 (20.2) 478 (19.4) Assistant Professor 46 (17.0) 49 (18.9) 52 (19.8) 338 (14.6) 358 (15.1) 379 (15.4)

Clinical Professor 6 (2.2) 6 (2.3) 6 (2.3) 54 (2.3) 63 (2.7) 67 (2.7) Clinical Associate Professor 18 (6.6) 22 (8.5) 18 (6.9) 129 (5.6) 155 (6.5) 174 (7.1) Clinical Assistant Professor 25 (9.2) 23 (8.9) 17 (6.5) 138 (6.0) 147 (6.2) 156 (6.3)

Instructor 1 (0.4) 1 (0.4) 2 (0.8) 47 (2.0) 35 (1.5) 26 (1.1) Teacher or Master Teacher 23 (8.Is this

the year whw5)

20 (7.7) 17 (6.5) 147 (6.4) 128 (5.4) 135 (5.5)

Visiting Professor 2 (0.7) 1 (0.4) 0 (0.0) 34 (1.5) 31 (1.3) 33 (1.3) Visiting Associate Professor 2 (0.7) 2 (0.8) 2 (0.8) 11 (0.5) 18 (0.8) 7 (0.3) Visiting Assistant Professor 11 (4.1) 5 (1.9) 8 (3.1) 27 (1.2) 35 (1.5) 37 (1.5)

Assistant Curator 53 (2.3) 54 (2.3) 56 (2.3) Language Lecturer 142 (6.1) 136 (5.7) 157 (6.4) Library Associate 10 (0.4) 8 (0.3) 7 (0.3)

TOTAL 271 259 262 2311 2377 2462


124

Table B.6 Comparison of Faculty Salary (Mean) for Steinhardt and the University, 2008-2010

Steinhardt University

Rank 2008-2009 2009-2010 2010-2011 2008-2009 2009-2010 2010-2011

N=271 N=259 N=262 N = 2311 N = 2377 N =2462

Professor $129,593

$129,920 $ 132,526 $170.700 $171..700 $ 175,900

Associate Professor $93,329

$92,216 $ 92,823

$103.700 $101.500 $ 103,800

Assistant Professor $71,528

$72,034 $ 73,009

$93.500 $92.700 $ 95,000

All Ranks $92,453

$92,305

$ 92,939

N/A N/A N/A

TABLE B.5 Comparison of Full-time Steinhardt and University Faculty by Gender & Ethnicity, 2008-2011

Gender Steinhardt University

Ethnicity 2008-2009 2009-2010 2010-2011 2008-2009 2009-2010 2010-2011 Female Male Female Male Female Male Female Male Female Male Female Male Black/African-

American 13 6 14 6 16 4 47 49 56 52 60 56

Caucasian 128 95 125 91 121 91 707 1070 731 1106 767 1133 Asian 15 5 9 6 13 6 75 97 88 102 92 101

Hispanic/Latino

International

4 5 3 5 4 7 46 46 44 52 4 57

61 104 47 97 46 90

TOTAL 160 111 151 108 154 108 937 1374 968 1409 1017 1445


125

APPENDIX C

Qualifications of the Faculty

TABLE C.1 Full-Time Teacher Education Faculty

2010-2011

Faculty profiles and bios are available at: http://steinhardt.nyu.edu/faculty_bios/list/Faculty/All Faculty teaching load will vary based on other responsibilities, i.e. serving as department chair or program director, supervising student teachers, administering research and program grants, etc. Listing of faculty scholarly publications are posted on the NYU Steinhardt Website at HTTP://STEINHARDT.NYU.EDU/FACULTY_BIOS/LIST/PUBLICATIONS and in faculty profiles available at Steinhardt’s password protected website (www.steinhardt/secure/teac) and will be made available during the TEAC site visit.

NAME RANK/TITLE DEGREE INSTITUTION YEAR FIELD EXPERTISE YEARS AT

NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Alter, Mark

Professor of Educational Psychology

Ph.D. Yeshiva University

1980 Special Education Special Education, instructional programming, teacher education, and professional development. NYS permanent special education teacher.

Over 25 years

4 during academic year, 2 summer

Beck, Sarah

Associate Professor

Ed.D. Harvard University

2002 Secondary English Education

Cultural contexts for literacy learning; equity issues in education; urban education; adolescent literacy

8 On sabbatical


126


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Benedict, Cathy

Assistant Professor Coordinator of Undergraduate Studies in Music Ed

Ed.D. Teachers College

Columbia University

2004 Music Education Social and critical theory contexts in education and music education. Music Education undergraduate advisor.

7 5

Bennison, Sarah

Visiting Assistant Professor of Social Studies

Ph.D. New York University

2006 History of Education

19th c. American women's history, religious history, and the history of the West

5 3

Blonstein, Jason

Master Teacher MAs

New York University

1970

Science Education

35 yrs. Teaching in NYC; Former Co-Founder & Co-Director, Cascades High School, a New Visions school; Teacher of the Year, NYC Alternative High Schools; Co-Founder, Alternative High Schools and Programs Professional Dev. Committee

8 6

MA

Fordham

1998

Administration & Supervision

Burgunder, Anne

Master Teacher of Math Education

MS Bank Street 1994 Secondary Math Education

Role of visual images in math education and various methods for teaching mathematics

5 3

Carothers, Suzanne

Professor Ph.D. New York University

1987 Curriculum & Instruction in Childhood

Over 30 years teaching experience in Early Childhood Education;

12 2


127


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Education Specialist in the pre-

school teaching of inner city and minority children.

Cohen, Robert

Professor of Social Studies Education

Ph.D. University of California/ Berkeley

1987 History Affiliated professor in History Department Metropolitan Studies. History of Education Social Studies Education Reform in New York City schools

12 2

Damast, Deborah

Master Teacher MA New York University

2002 Dance Education Dance Education 10 3

Darts, David

Assistant Professor of Art Education

Ph.D. University of British Columbia

2004 Studio Art Convergences between education, contemporary art and media, technology and democracy; Media literacy in art classroom, research in art education studio courses in Venice

5 2

Desai, Dipti

Associate Professor of Art and Art Education

Ph.D. University of Wisconsin-Madison

1989 Art Education Has been teaching art for 20 years and involved in interdisciplinary art education curriculum development and assessment. Publication record in prominent art

12

5


128


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY education journals and published/ exhibited original artworks. Nominated for the Manual Barkan Award for scholarly contribution to the field of art education.

Doucet, Fabienne

Assistant Professor of Education

Ph.D. University of North Caroline Greensboro

2000 Early Childhood Education

Educational experiences of immigrant and U.S.-born children of color and their families

6 4

Eisenstein Ebsworth, Miriam

Associate Professor

Ph.D. CUNY 1979 Linguistics TESOL, Foreign Language Education & Bilingual Education, Applied Linguistics

26 4

Elliott, David

Professor Ph.D. Case Western Reserve University

1983 Music Education Philosophy of music, music cognition, curriculum development, multicultural music, research methods

9 6

Fleisher, Lisa

Associate Professor

Ph.D. University of Illinois

1979 Special Education Coordinator of the Early Childhood Special Education Program.

26 4 during academic

year, 1 summer

Fraser-Abder, Pamela

Associate Professor Director of Science

Ph.D. Pennsylvania State University

1982 Science Education Science Education; Director of Mathematics Science Technology Enhancement Program

21 2


129


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Education (MSTEP)

Friedlander, Helen


1970 Special Education NYS & NYC Special Education (K-12); NYS & NYC School Administration and Supervision; program development.

9 4

Gilbert, John

Associate Professor

Ed.D. Columbia University

1969 Music Education, Musicology, Composition

Creative Process in Music Education, Technological Trends in Music Ed. Interdisciplinary study, Collaborative Education, Dissertation Proposal Seminar, Research Design and Method.

41 11

Gottlieb, Jay Professor Ph.D. Yeshiva University

1972 Psychological Educational Research in Special Education

Certified Teacher of Biology, General Science, & primary school; Educational Evaluator, researcher for various educational institutes including the US Office of Education, Bureau of Education for the Handicapped.

26 5

Gouck, Maura


1970 English Literature 30 years secondary English teaching experience in NYC.

8 3


130


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Supervises Teaching Fellows

Green, Judith


2003 Math Education Use of technology in mathematics education

2 6

Hull, Glynda

Professor of English Education

Ph.D. University of Pittsburgh

1983 English Education Literacy development in the context of digital technologies and globalization

2 On Leave

Jacobs, Benjamin M.

Assistant Professor of Social Studies Education and Jewish studies

Ph.D. Teachers College

2005 Teacher Education and Jewish Studies

Social Education, Jewish Education, history of education, curriculum studies, & education of ethnic groups

1 2

Jeffery, Jill V.

Visiting Assistant Professor of English Education and Literacy


2010 English Education and Literacy Education

Teaching and assessment of writing using integrated methodological approaches

1 3

King, Karen

Associate Professor

Ph.D. University of Maryland

1997 Mathematics Mathematics Education

5 On Leave

Kirch, Susan A.

Associate Professor of Science Education

Ph.D. Harvard University

1996 Science Education Investigations of teaching and learning, science in urban elementary schools, and studies of teacher learning in the areas of science & inclusion

3 4


131


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Kirkland, David E.

Assistant Professor of English Education

Ph.D. Michigan State University

2006 English and Urban Education

Urban youth culture, language and literacy, urban teacher preparation, and digital media

4

5

Koff, Susan Clinical Associate Professor

Ed.D. Temple University

1995 Dance Education Dance Education, Research, Professional Development

6 6

Krasnow, Maris

Clinical Associate Professor

Ed.D. Teachers College, Columbia

1982 Curriculum & Teaching

Early Childhood , Special Education, Urban Education and Mentoring New Teachers.

12 6

Labov, Joanna

Clinical Assistant Professor in TESOL

Ph.D. University of Pennsylvania

2000 TESOL Preparation of student teachers to teach ESL, the effectiveness of ESL pedagogy & methods to improve second language pronunciation

2 3 (on leave in fall 2010)

Leou, Mary Clinical Associate Professor Director of the Wallerstein for Urban Environmental Education

Ed.D.

Teachers College, Columbia

1997 Curriculum & Teaching

Teacher Development (pre-service & in-service), urban environmental education, school reform, experiential learning

10 4

Light, Rebecca

Visiting Assistant Professor


2010 Childhood Education

Informal language used by early childhood teachers in preschool

1 2


132


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY settings Llosa, Lorena Assistant

Professor Ph.D. UCLA 2005 TESOL/Bilingual

Education Testing, Research, Second Language Acquisition

5 5

Magill, Richard

Professor, Acting Chair of the Department of Teaching and Learning

Ph.D. Florida State University

1974 Educational Psychology

Human learning and performance

4 2

Malczewski, Joan

Assistant Professor of History and Social Studies

Ph.D. Columbia University

2002 History and Social Studies Education

Relationship between northern philanthropy and southern education history

4 4

McCallister, Cynthia

Associate Professor

Ed.D. University of Maine

1995 Literacy Education

Over 20 years experience as classroom & college teacher specializing in the application of literacy acquisition to teaching practice

7 3

McDonald, Elizabeth

Master Teacher CAS Harvard University

1991 Administration 27 years of classroom teaching, Elementary administration, & professional development

10 4

McDonald, Joseph

Professor Ed.D. Harvard University

1986 English Education School partnerships and reform

10

4


133


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Milne, Catherine

Associate Professor

Ph.D. .

Curtin University of Technology Perth, Australia

1988 Science Education The role of history and philosophy of science in school science; learning science in urban schools

8

2

Montgomery, David

Clinical Assistant Professor


2006 Educational Theatre

Drama Education, theatre for young audiences, student teaching and integrated arts

5 6

Mulligan, Carole

Master Teacher MA St. John’s Graduate Institute

1978 Liberal Arts 17 years experience teaching pk-6 grade in NYC. Early Childhood & Mathematics Education

9 6

Nero, Shondel

Associate Professor

Ed.D. Teachers College

1997 MMS/TESOL Education of second language and second dialect speakers

3 4

Noguera, Pedro

Professor Director, Metro Center on Urban Education

Ph.D. UC: Berkeley

1989 Sociology Urban Education, minority education

8 2

O’Connor, Erin

Assistant Professor

Ed.D. Harvard University

2005 Early Childhood Education

Relationships with mothers and teachers and the impacts of these relationships on children’s development in early and middle childhood

5 1


134


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Pietanza, Rosa

Master Teacher

MA

Hunger College,

1975

Italian

24 years of teaching and administration. Coordinator School Partnerships

5 4

CAS

Brooklyn CollegE

1981 Administration & Supervision

Pitts, Harriet


Ed.D. Rutgers University


Literacy 5 4

Rafter, Joseph



1993 Psychology Over 30 years as an elementary school teacher in NYC.

10 4

Romandetto, Patricia

Master Teacher Assistant Director, Office of Clinical Studies

MS Lehman College


Administration and Supervision

5 2

Rosenberg, Joan


Ed.D. Teachers College, Columbia University

1994 Sp Ed/Admin Special Education. Pre-service Teacher Development

14 4

Salvatore, Joseph

Teacher MFA University of. Massachusetts, Amherst

1998 Theatre, Dramaturgy/ Directing

Artistic/Education director for non-profit youth theatre company, curriculum development, active director in theatre in NY & NJ. Received James Baldwin Playwriting Award, 1998

9 6 For pre-service Theatre

Educators


135


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Schiffman, Howard S.

Visiting Associate Professor

Ph.D. University of Wales

2006 Environmental Education

The law of the sea, marine conservation law and policy and international dispute settlement

1 3

Schwartz, Barbara

Clinical Associate Professor


1987 Educational Psychology with Special Education Focus

35 years in the field of Early Childhood Special Education. Served as an Adjunct Instructor in Ed Psych beginning in 1978 and Full-Time from 1990 to present.

11 4

Simon, Martin

Professor Ph.D. University of Massachusetts at Amherst

1986 Elementary Math Education

Development of mathematics teachers as they learn to teach mathematics with a conceptual focus

5 2

Smithner, Nancy

Teacher Ph.D. New York University


20 years directing experience. Expertise in Physical theatre, acting, mime, and movement research. Integration of text, movement, and music

19 5

Stahl, Katherine Dougherty

Assistant Professor

Ed.D. University of Georgia

2003 Literacy Reading acquisition and reading comprehension in the elementary years

5 6

Tang, Lixing Clinical Ph.D. New York 1984 Secondary Over 35 years teaching 10 2


136


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY (Frank) Professor

Director, Multilingual Multicultural Studies

University TESOL English as a second/ foreign language in secondary schools and colleges in China and the U.S.

On

sabbatical in fall 2010

Taylor, Philip Program Director, Associate Professor of Educational Theatre



Qualitative research, reflective praxis, drama and arts education, applied theatre

4 4

Tobias, Robert

Clinical Professor Director of the Center for Research on Teaching and Learning

MA Temple University

1969 Psychology 33 years as teacher, researcher, and Assessment Specialist for NYC public schools. Retired in 2001 as Executive Director of Assessment and Accountability. Facilitates research and evaluation of Teacher Education program

8 2

Turk, Diana Associate Professor

Ph.D. University of Maryland

1999 American Studies US History, women’s history, educational and historical methods, teaching with technology

8 4

Vukovic, Rose

Assistant Professor

Ph.D.

University of British Columbia

2006

Special Education & School Psychology

Math difficulties & How math difficulties develop in children

4 4


137


NYU

# TEACHER ED.

COURSES TAUGHT

ANNUALLY Zaslavsky, Orit

Professor Ph.D. Technion: Haifa

1987 Math Education Development of teacher-educators, the nature of productive mathematics-related tasks and examples

2 4

TABLE C.2 Adjunct Faculty 2010-2011 and TABLE C.3 Supervisors 2010-2011 will be made available on the NYU Steinhardt’s password protected website (www.Steinhardt/secure/TEAC). and copies are available for the site visit.


138

Appendix D Program Requirements

UNDERGRADUTE 1. Admissions

Admissions to the NYU Steinhardt Teacher Education Program for undergraduate students is administered centrally by NYU, and requirements are posted at http://www.nyu.edu/admissions/undergraduate-admissions/applying-for-admission/freshman- applicants/general-requirements.html. The capacity for successful undergraduate work is measured through careful consideration of secondary school records; the application essay; recommendations from guidance counselors, teachers, and others; and scores on standardized tests (SAT 1 & 2 or ACT). Participation in meaningful school and community activities is an important factor. NYU actively seeks students who are varied in interests, talents and goals, as well as in social and economic backgrounds.

Admissions to the Teacher Education Program for transfer students is administered at Steinhardt http://steinhardt.nyu.edu/undergraduate_transfer/

Undergraduate admissions statistics for NYU, including graduation and retention rates are available at http://www.nyu.edu/admissions/undergraduate-admissions/is-nyu-right-for- you/faqs.html 2. Program Requirements and Standards

The teacher education program offers pre-service curricula at the baccalaureate degree level leading to initial certification in Childhood, Early Childhood, and Special Education, Educational Theatre, English Education, Foreign Language Education, Mathematics Education, Music Education, Science Education: Biology, Chemistry, Earth Science, Physics and Social Studies Education.

The Teacher Education Requirements reported below are also listed in the Undergraduate Bulletin 2010-2012 available at http://steinhardt.nyu.edu/bulletin/

3. Course titles and descriptions The Plan of Study for all the baccalaureate curricula areas leading to initial certification are listed in Table D1.1

4. Graduation requirements The school-wide requirement is that students must successfully complete all course

requirements with a general grade point average of at least 2.0 for graduation with a baccalaureate degree. Additional requirements may be imposed by each program of study. Teacher education students meet the same requirements as other Steinhardt students.


139

Graduation requirements are provided at http://steinhardt.nyu.edu/registration/standards. The Registrar’s Office Degree Audit and Graduation Services checks for the completion of requirements as outlined on the students’ program of study: http://www.nyu.edu/registrar/transcripts-certification/degree-progress.html and will not confer any degree if all course work and credit requirements for graduating students as outlined by program of study are not satisfied.

GRADUATE 1. Admissions

Admissions to the graduate Teacher Education Program is administered at Steinhardt and an overview is provided at: http://steinhardt.nyu.edu/graduate_admissions/. Application guidelines for all graduate programs, including fast track are listed at http://steinhardt.nyu.edu/graduate_admissions/guide. Steinhardt School requirements for graduate students are listed at http://steinhardt.nyu.edu/graduate_admissions/guide/masters Content area requirements for admissions to graduate teacher education certification areas are listed at following website: http://steinhardt.nyu.edu/scmsAdmin/uploads/000/779/TeacherCertificationRequirements.pdf . 2. Program Requirements and Standards pdf

Program requirements are stated in the Steinhardt Graduate bulletin at: http://steinhardt.nyu.edu/steinhardt/bulletin/nyu_steinhardt_graduate_bulletin_2010_info.

3. Course titles and descriptions The Plan of Study for all the graduate curricula areas leading to teacher certification are listed in Table D1.1 4. Graduation requirements

School-wide requirements are that students must successfully complete all degree requirements. A scholastic average of at least 2.5 for both the total record and for courses in specialization is required for graduation. Additional requirements may be imposed by programs of study. Teacher education students meet the same requirements as other Steinhardt students. Graduation requirements are provided at http://steinhardt.nyu.edu/registration/standards. The Registrar’s Office Degree Audit and Graduation Services checks for the completion of requirements as outlined on the students’ program of study: http://www.nyu.edu/registrar/transcripts-certification/degree-progress.html and will not confer any degree if all course work and credit requirements for graduating students as outlined by program of study are not satisfied.


140

NYS TEACHER CERTIFICATION REQUIREMENTS New York State Teacher Certification Requirements listed below are noted in the

Steinhardt Graduate Bulletin for 2009-2011 http://steinhardt.nyu.edu/steinhardt/bulletin/nyu_steinhardt_graduate_bulletin_2010_info.pdf TEACHER CERTIFICATON PROGRAM AT NYU

The requirements for registration of curricula in teacher education are outlined at the NYS Education Department Office of Higher Education website: http://www.highered.nysed.gov/ocue/52.21.htm and as noted in Appendix A and Appendix B, the NYU teacher certification program curricula areas are registered with the NYSED and are listed in the Inventory of Registered Programs available at http://www.nysed.gov/heds/IRPSL1.html.

The NYU Teacher Education Program adheres to requirements stipulated by the NYS Office of Teaching Initiatives and to the Regulations of the NYS Commissioner of Education: http://www.highered.nysed.gov/tcert/regulations.html and the Pedagogical Core requirements as outlined in the NYSED Higher Education website: http://www.highered.nysed.gov/ocue/aipr/documents/pedcoregeneral-Dec2010.doc

Table D.1.1 lists the NYU teacher certification curricula areas with the links to the corresponding 2010-2011 plans of study. Each plan of study lists course requirements for the Morse Academic Plan (liberal arts and science required courses at the undergraduate level) and lists the content core courses –that correlate to TEAC Principle I - 1.1 Subject Matter knowledge, as well as the general pedagogical core, specialized pedagogical core, fieldwork and student teaching and practica, and the culminating/terminal experience (graduate level).

Table D.1.2 available on the Steinhardt’s password protected website (www.Steinhardt/secure/TEAC) and in the TEAC coordinators’ office lists the teacher certification areas and specific courses that meet NYS general pedagogical core requirements, specialized pedagogical core requirements and the fieldwork requirements. We have correlated these courses to TEAC Principle I (1.2, 1.3, 1.4.1, 1.4.2 and 1.4.3).


141

Table D.1.1 Graduate and Undergraduate NYS Registered Teacher Education Curricula Options with credit and course requirements

(Content Core and Pedagogical Core)

GRADUATE

Curricular Area Program of Study

Teaching Educational Theatre, All Grades http://steinhardt.nyu.edu/music/edtheatre/curriculum/graduate/edta Teaching Music, All Grades http://steinhardt.nyu.edu/music/education/curriculum/graduate Teaching Dance, All Grades http://steinhardt.nyu.edu/music/dance/curriculum/all_grades Teaching Art, All Grades http://steinhardt.nyu.edu/art/education/curriculum Childhood Education http://steinhardt.nyu.edu/teachlearn/early_childhood/ma/program_of_stu

dy/ Early Childhood Education http://steinhardt.nyu.edu/teachlearn/early_childhood/ma/program_of_stu

dy/ Teaching English, 7-12 http://steinhardt.nyu.edu/teachlearn/english/ma/program_of_study Teaching a Foreign Language 7-12 (Chinese, French, German, Hebrew, Italian, Japanese, Latin, Russian, or Spanish)

http://steinhardt.nyu.edu/teachlearn/foreign/ma/program_of_study#with _certification

Teaching Biology 7-12 http://steinhardt.nyu.edu/teachlearn/science/ma/program_of_study

Teaching Chemistry 7-12 http://steinhardt.nyu.edu/teachlearn/science/ma/program_of_study Teaching Physics 7-12 http://steinhardt.nyu.edu/teachlearn/science/ma/program_of_study Teaching Mathematics 7-12 http://steinhardt.nyu.edu/teachlearn/math/ma/program_of_study Teaching Social Studies 7-12 http://steinhardt.nyu.edu/teachlearn/social_studies/ma/program_of_stud

y Bilingual Education for Teachers http://steinhardt.nyu.edu/teachlearn/bilingual/ma/program_of_study Literacy (B-6, 5-12) http://steinhardt.nyu.edu/teachlearn/literacy/ma/program_of_study Teachers of English to Speakers of Other Languages (TESOL)

http://steinhardt.nyu.edu/teachlearn/tesol/ma/program_of_study#with_ce rtification

Special Education in Childhood http://steinhardt.nyu.edu/teachlearn/special/ma/childhood#sech

Special Education in Early Childhood http://steinhardt.nyu.edu/teachlearn/special/ma/early_childhood#seec Dual Certification: Educational Theatre, All Grades, with English Education 7-12

http://steinhardt.nyu.edu/music/edtheatre/curriculum/graduate/eted


142

Educational Theatre, All Grades, with Social Studies Education 7-12

http://steinhardt.nyu.edu/music/edtheatre/curriculum/graduate/etss

Teaching a Foreign Language 7-12 (Chinese, French, German, Hebrew, Italian, Japanese, Latin, Russian, or Spanish) with TESOL;

http://steinhardt.nyu.edu/teachlearn/foreign/ma/program_of_study#dual

Childhood Education/Childhood Special Education; http://steinhardt.nyu.edu/teachlearn/special/ma/childhood#csec

Early Childhood Education/Early Childhood Special Education

http://steinhardt.nyu.edu/teachlearn/special/ma/early_childhood#esee

UNDERGRADUATE

Curricular Area Program of Study

Educational Theatre, All Grades http://steinhardt.nyu.edu/music/edtheatre/curriculum/undergraduate

Teaching Music, All Grades http://steinhardt.nyu.edu/music/education/curriculum/undergraduate

Teaching English 7-12 http://steinhardt.nyu.edu/teachlearn/english/bs/program_of_study

Teaching a Foreign Language http://steinhardt.nyu.edu/teachlearn/foreign/bs/program_of_study

Teaching Biology 7-12 http://steinhardt.nyu.edu/teachlearn/science/bs/program_of_study#bio

Teaching Chemistry 7-12 http://steinhardt.nyu.edu/teachlearn/science/bs/program_of_study#chem

Teaching Physics 7-12

http://steinhardt.nyu.edu/teachlearn/science/bs/program_of_study#physi cs

Teaching Earth Science 7-12 http://steinhardt.nyu.edu/teachlearn/science/bs/program_of_study#earth

Dual Certification-Childhood Ed./Childhood Special Ed.

http://steinhardt.nyu.edu/teachlearn/childhood/bs/program_of_study

Dual Certification-Early Childhood Ed./Childhood Special Ed.

http://steinhardt.nyu.edu/teachlearn/early_childhood/bs/program_of_stu dy

Teaching Mathematics 7-12 http://steinhardt.nyu.edu/teachlearn/math/bs/program_of_study

Social Studies: 7-12 http://steinhardt.nyu.edu/teachlearn/social_studies/bs/program_of_study


143

* All Course Codes are being updates beginning summer 2011. NYU is transitioning to PEOPLESOFT- a new Student Information System. Table D 1.2. NYS Pedagogical Core Requirements for Certification in Teacher Education is available on the Steinhardt password protected website (www.Steinhardt/secure/TEAC) and will be made available to TEAC auditors during the site visit). Table D.1.2 identifies curricula area courses that meet the NYS Pedagogical Core Requirements (general requirements, program specific requirements, and field requirements) and corresponding TEAC Quality Principle I (1.2 Pedagogical knowledge, 1.3 Caring and effective teaching skill, 1.4.1 Cross-cutting theme: Learning how to learn, 1.4.2 Cross-cutting theme: Multicultural perspectives and 1.4.3 Cross-cutting theme: Technology. We included course titles only. All NYU course numbers are changing as of summer 2011. Course Descriptions are available at: http://steinhardt.nyu.edu/courses/


144

Appendix E: Inventory: Status of evidence from measures and indicators for TEAC Quality Principle I

Type of Evidence Available and in the Brief Not Available and Not in the Brief

Note: items under each category are examples.

Program may have more or different evidence

Relied on Reasons for including the results

in the Brief (Location in Brief)

Not relied on Reasons for not relying

on this evidence (Location in the Brief)

For future use

Reasons for including in future Briefs

Not for future use

Reasons for not including in future Briefs

Grades

1. Student grades and grade point averages

Content Knowledge GPA, Pedagogical Knowledge GPA, Teaching Skills GPA, and Cross-Cutting Theses GPA are valid and reliable measures of student mastery of the skills and knowledge that are associated with the claims. (pp 44-46)

Scores on standardized tests

2. Student scores on standardized license or board examinations

Scaled scores on the NYSTCE Content Specialty Tests and Assessment of Teaching Skills-Written exams are valid, reliable, and sensitive measures of Content Knowledge and Pedagogical Knowledge, while scaled scores on the Liberal Arts and Sciences Test are valid measures of the cross-cutting theme of Learning-to-Learn, which requires a broad and deep understanding of the tools and concepts of the liberal arts and sciences. (pp 38-41)

3. Student scores on undergraduate and/or graduate admission tests of subject matter knowledge and aptitude

NYU’s claim of Content Knowledge pertains to the knowledge of program completers. Faculty believes that admissions tests for undergraduates taken four or more years prior to graduation are not valid measures of the claim because they are distal in time and not well aligned with the constructs in content. Admissions tests are optional for graduate admissions and few students submit them.


145

4.Standardized scores and gains of the program graduates’ own students

NYU used the VAM test score gains of the pupils of graduates teaching in grades 4-8 in the NYC public schools to measure Clinical Competence. We provide evidence that VAM scores are moderately valid and reliable measures of teacher effectiveness, which is a construct that is aligned with Clinical Competence. (pp 54-55)

Ratings

5. Ratings of portfolios of academic and clinical accomplishment

Portfolio data were not included in the original Brief and will not be used in this Brief. Attempts to develop a standard rubric to score portfolios proved unsuccessful due to insufficient inter-rater agreement.

6. Third-party rating of program's students

NYU considered using third-party ratings of program students but determined the procedures to be not feasible logistically. However, the faculty considers this to be valuable additional evidence and will attempt to design feasible methods in the future.

7. Ratings of in-service, clinical, and PDS teaching

An important measure used to assess all four claims and the cross-cutting theme of Learning-to-Learn is the DRSTOS-R. This observation protocol is used by field supervisors to assess the developing pedagogical proficiency of student teachers in clinical practice. Evidence of empirical validity and reliability is presented in the Brief. (pp. 32-37)

NYU believes that in-service ratings of the teaching of its graduates can provide useful data for reflecting back upon the quality of graduates’ program preparation. Efforts to develop methods to do this have been complicated by negotiations between the NYC and NYS Departments of Education and respective collective bargaining agencies. NYS has passed a new law that will lead to the development of a common teacher evaluation system across the entire state. NYU is hopeful that this system will provide in-service data that can be used in future inquiries.


146

8. Ratings by cooperating teacher and college/ university supervisors, of practice teachers' work samples

Student teachers’ work samples are used as an important source of evidence for DRSTOS-R assessments. The work samples include journals, lesson plans, written reflections on practice, and pupil work. Field supervisors review the work samples and then use them holistically to arrive at the ratings of related DRSTOS-R items. This evidence is cited in the protocols completed by the field supervisors. (pp. 32-37)

Rates

9. Rates of completion of courses and program

The faculty believes these data are not valid measures of the claims and, therefore, they are not included in the Brief.


147

10. Graduates' career retention rates

NYU was able to obtain data from its Graduate Tracking Study to compute retention rates up to three years for graduates teaching in the NYC public schools. The data were obtained for a large sample of 1,108 graduates from the classes of 2004-08. These data are reliable and valid for assessing the claim that graduates are Caring Professionals who have the commitment and skill to sustain their careers in inner-city schools. (pp. 51-54)

11. Graduates' job placement rates

Job placement rates are reported in this Brief based on data from the Graduate Tracking Study and One-Year Follow-Up Survey. These data are not used to support the claims, since they are subject to the vicissitudes of the job market. Accordingly, they are used descriptively but not tested against any program standard. (pp.49-54)

12. Rates of graduates' professional advanced study

NYU collected these data in its Program Exit Surveys for 2009 and 2010. Faculty believes additional data from future surveys will be needed in order to generate reliable estimates of rates of professional advanced study.

13. Rates of graduates' leadership roles

NYU will be collecting these data in a planned Five-Year Follow-Up Survey and they will appear in future reports.

14. Rates of graduates' professional service activities



148

Case studies and alumni competence

15. Evaluations of graduates by their own pupils

NYU believes that the questionable reliability and validity of these data render the high resource expenditures required to collect them unwarranted.

16. Alumni self-assessment of their accomplishments


17. Third-party professional recognition of graduates (e.g. NPTS)


18. Employers' evaluations of the program's graduates

Principals’ ratings of all teachers will be part of the new NYS teacher evaluation system. NYU plans to obtain these data for its graduates and use them in future studies.

19. Graduates' authoring of textbooks, curriculum materials, etc.


20. Case studies of graduates’ own pupils’ learning and accomplishment

NYU believes the cost of collecting these data would be excessive and the inferences that might be drawn from them concerning graduates’ effectiveness would have weak validity.

Other Data


149

21. Students’ self-ratings of growth during student teaching.

NYU uses the ETFQ to assess student teachers’ perceptions of growth in Content Knowledge, Pedagogical Knowledge, and Clinical Skills. The results of this assessment have theoretical validity and have been consistent across many cohorts. (pp. 41-42)

22. Students’ dispositions to teaching.

NYU has developed EBMAS, a survey that assesses students’ self perceptions of general teaching efficacy, personal teaching efficacy, and social justice/multicultural attitudes. EBMAS has demonstrated empirical validity and internal consistency reliability for measuring these dispositions which research has linked to teacher quality. (pp. 42-44)


150

23. Graduates ratings of the their preparation for teaching

NYU conducts two surveys of teacher-education program graduates: the Program Exit Survey and the One-Year Follow-Up Survey. These surveys assess the extent to which graduates feel that the program has prepared them to be successful teachers. The surveys show consistency of results for successive administrations, convergence of findings between the two surveys, and consistency with the results from a source survey developed by Arthur Levine. In addition, the items are well aligned with NYU’s claims. (pp. 46-51)

24. Demographics of graduates’ schools of employment

Through its electronic graduate tracking study, NYU assesses the demographic characteristics of the NYC public schools in which graduates are employed. These data are used to assess the graduates’ commitment to working in inner-city schools, which is aligned with the claim of Caring Professionals (pp. 51-54)


151

Appendix F Local Assessments

F-1 Domain Referenced Student Teacher Observation Scale-Revised (DRSTOS-R) F-2 Educational Beliefs and Multicultural Attitudes Survey (EBMAS) F-3 End of Term Feedback Questionnaire for Student Teachers (ETFQ) F-4 Program Exit Survey 2010 F-5 First Year Teacher: Feedback Form


152

Domain Referenced Student Teacher Observation Scale - Revised

(DRSTOS-R) Student Teacher & Placement Information

Please check one: Junior Senior

Undergraduate Fast Track Regular Track

Graduate

Major/Program(s):

Certification track? Yes No

Native English Speaker?

Yes

No

Placement (check one)

1 out of 4 2 out of 4 3 out of 4 4 out of 4

1 out of 2 2 out of 2

* Early Childhood Majors Only 1 out of 3 2 out of 3 3 out of 3

Placement Information

General Education

Self-Contained Special Education

CTT

0 - 25% English Language Learners

26 - 50% English Language Learners

51%+ English Language Learners

Grade(s) Content/Specialty Area (if applicable)

School Name/PS #

Cooperating Teacher

Last

First

Additional Notes on Placement (ex: push-in, pull-out, SETTS/Resource Room)

LEVEL OF PERFORMANCE


153

ELEMENT

NOT YET PROFICIENT

PARTIALLY PROFICIENT ENTRY LEVEL PROFICIENT

PROFICIENT EVIDENCE

PLANNING AND PREPARATION

1. PEDAGOGICAL CONTENT KNOWLEDGE

Student teacher displays inadequate understanding of pedagogical issues involved in pupil learning of the content.

Student teacher displays basic content knowledge but does not articulate connections among content, pedagogy, and pupil development.

Student teacher displays sufficient content knowledge but does not sufficiently articulate connections among content, pedagogy, and pupil development.

Pedagogical practices reflect current research on best pedagogical practice within the discipline and the anticipation of potential pupil misconceptions. Student teacher makes connections with or to other disciplines.

1 2 3 4

2. KNOWLEDGE OF CONTENT STANDARDS

Student teacher displays inadequate evidence of familiarity with content standards.

Student teacher displays basic knowledge of content standards, without evidence of connecting to standards beyond the current lesson.

Student teacher displays a sufficient understanding of the city/state content standards and makes connections to other standards within and/or beyond content area.

Student teacher displays a strong understanding of the city/state content standards and makes connections to other standards within and/or beyond content area.

1 2 3 4

3. LONG/SHORT TERM PLANNING

Planning for instruction is not connected to longer-term goals or to the pedagogical content knowledge of the subject, the pupils, or the standards, and are unclear to most pupils in the class.

Planning for instruction is partially connected to longer-term goals and there is limited use of pedagogical content knowledge of the subject, the pupils, or the standards.

Planning for instruction connects to longer-term goals and sufficiently uses pedagogical content knowledge of the subject, the pupils, or the standards.

Planning for instruction connects to longer-term goals and effectively uses pedagogical content knowledge of the subject, the pupils, or the standards.

1 2 3 4


154

ELEMENT LEVEL OF PERFORMANCE

NOT YET PROFICIENT

PARTIALLY PROFICIENT ENTRY LEVEL PROFICIENT

PROFICIENT EVIDENCE

4. CONSTRAINTS ON TEACHING AND LEARNING

Student teacher plans and teaches without regard to the particular possibilities and limits of his/her classroom context.

Student teacher understands some of the curricular and resource possibilities and constraints of the context but does not effectively use them in planning or teaching.

Student teacher sufficiently understands the curricular and resource possibilities and constraints of the context and begins to use them in planning or teaching.

Student teacher thoroughly understands the curricular and resource possibilities and constraints of the context and uses them effectively in planning or teaching.

1 2 3 4

5. CRITERIA AND STANDARDS

The proposed approach contains no clear criteria or standards.

Assessment criteria and standards are unclear.

Assessment criteria and standards are generally appropriate and sufficiently clear.

Assessment criteria and standards are well developed and explicit.

1 2 3 4

6. FEEDBACK, REFLECTION AND USE FOR PLANNING

Information from assessments (tests, observations, conferences, etc.) affects planning for these pupils only minimally.

Student teacher uses assessment results to plan for the class as a whole.

Student teacher uses assessment results to plan for individuals and groups of pupils as well as the class as a whole.

Student teacher uses assessment results to plan for individuals and groups of pupils as well as the class as a whole and uses pupil input in assessment planning.

1 2 3 4


155

CLASSROOM ENVIRONMENT

7. STUDENT TEACHER INTERACTION WITH PUPILS

Student teacher’s voice controls the classroom environment. Students’ thoughts need to be nurtured and validated.

Student teacher is beginning to elicit students’ thoughts in the classroom environment.

Student teacher regularly provides students with a venue to share their thoughts and ideas.

The classroom environment reflects a balance of student teacher’s and students’ thoughts. Students’ thoughts are nurtured and encouraged.

1 2 3 4

8. CLASSROOM INTERACTION

Classroom interactions are frequently characterized by conflict, sarcasm, or put- downs.

Classroom interactions are occasionally characterized by conflict, sarcasm, or put-downs.

Classroom interactions are generally polite and mutually respectful.

Classroom functions as a genuinely polite, caring and mutually respectful community.

1 2 3 4

9. FUNCTIONING OF LEARNING GROUPS

Pupils not working with the student teacher are not productively engaged in the task(s). Students in groups are off-task or are working independently.

Tasks for group work are partially organized, resulting in some off-task behavior when student teacher is involved with one group. Students sit together to work but interact minimally.

Tasks for group work are organized, and groups are managed so most pupils are engaged most of the time. Student teacher facilitates interaction between group members.

Tasks for group work are well organized, and groups are managed so most pupils are engaged at all times and are working collaboratively.

1 2 3 4

10. TRANSITIONS

Much time is lost during transitions.

Transitions are sporadically efficient, resulting in some loss of instructional time.

Transitions mostly occur smoothly, with minimal loss of instructional time.

Transitions occur smoothly, with almost no loss of instructional time.

1 2 3 4


156

11. MATERIALS AND SUPPLIES

Materials are handled inefficiently, resulting in significant loss of instructional time.

Routines for handling materials and supplies are sporadically efficient, resulting in some disruption of instruction.

Routines for handling materials and supplies are mostly efficient, with minimal disruption of instruction.

Routines for handling materials and supplies are consistently efficient.

1 2 3 4

12. MUTUAL EXPECTATIONS

No standards of conduct appear to have been established, or pupils are confused as to what the standards are.

Standards of conduct appear to have been established for most situations, and most pupils seem to understand them.

Standards of conduct are clear to all pupils.

Standards of conduct are clear to all pupils, and there is evidence of some student participation in their formulation.

1 2 3 4

13. AWARENESS OF PUPIL BEHAVIOR

Pupil behavior is not monitored, and student teacher is unaware of what pupils are doing.

Student teacher is generally aware of pupil behavior but misses the activities of some pupils.

Student teacher is alert to pupil behavior most of the time.

Student teacher is alert to pupil behavior at all times and pupils participate in the monitoring process.

1 2 3 4 INSTRUCTION

14. LESSON STRUCTURE AND TIME MANAGEMENT

The lesson has no clearly defined structure. The pace of the lesson is too slow, or rushed or both. Classroom time is not spent on instruction or there is significant loss of instructional time.

The lesson has a recognizable structure, although it is not uniformly maintained throughout the lesson. Pacing of the lesson is inconsistent. There is some loss of instructional time.

The lesson has a clearly defined structure around which the activities are organized. Pacing of the lesson is generally appropriate with minimal loss of instructional time.

The lesson’s structure is highly coherent, so that there is almost no loss of instructional time. Pacing of the lesson is appropriate for all students.

1 2 3 4


157

15. CLARITY OF GOALS

Goals are inappropriately selected and are not suitable for most pupils.

Goals are appropriately selected and partially suitable for most pupils.

Goals are sufficiently selected in their content and level of expectations and are suitable for most pupils in the class.

Goals are highly sufficient in their selection of content and level of expectations and are suitable for most pupils in the class.

1 2 3 4 16. KNOWLEDGE OF

STUDENTS: PUPILS’ SKILLS , CULTURAL HERITAGE, KNOWLEDGE, INTERESTS, LEARNING STYLES INSTRUCTIONAL NEEDS

Student teacher demonstrates an inadequate knowledge of pupils’ skills, knowledge and learning styles, and does not indicate that such knowledge is valuable.

Student teacher recognizes the value of understanding pupils’ skills, knowledge and learning styles, but displays this knowledge for the class only as a whole and rarely for those with special needs.

Student teacher demonstrates a sufficient knowledge of pupils’ skills, knowledge and learning styles for groups of pupils including those with special needs and recognizes the value of this knowledge.

Student teacher demonstrates a strong knowledge of pupils’ skills, knowledge and learning styles for groups of pupils and recognizes the value of this knowledge including those with special needs.

1 2 3 4

17. STUDENT TEACHER/ PUPIL COMMUNICATIONS

Student teacher’s or pupils’ spoken language is inaudible, or written language is illegible. Spoken or written language may contain many grammar and syntax errors. Vocabulary may be inappropriate, vague, or used incorrectly, leaving pupils confused.

Student teacher’s or pupils’ spoken language is audible, and written language is legible. Both are used correctly. Student teacher vocabulary is correct but limited or is not appropriate to pupils’ ages or backgrounds.

Student teacher’s and pupils’ spoken and written language are sufficiently clear and appropriate to pupils’ age and interests.

Student teacher’s spoken and written language is clear, correct, and enhances the learning of the subject. Pupils are mastering the standard written language as writers and readers.

1 2 3 4

18. DISCUSSION STYLE

Interaction between student teacher and pupils is predominantly recitation style, with student teacher mediating all questions and answers.

Student teacher attempts to engage pupils in discussion, with uneven results.

Most classroom interaction represents discussion, with student teacher taking a facilitating role.

Classroom interaction represents discussion, with student teacher stepping, when appropriate, to the side so pupil-pupil talk dominates.


158

1 2 3 4

PROFESSIONAL RESPONSIBILITIES 19. RELATIONSHIPS

WITH ADULTS: SUPERVISOR, COOPERATING TEACHER, TEACHERS, SCHOOL STAFF, & PARENTS/ GUARDIANS.

Student teacher’s relationships with adults are negative or self- serving.

Student teacher maintains cordial relationships with adults.

Support and cooperation characterize relationships with others.

Student teacher is able to maintain positive relationships with adults and functions effectively as part of a team.

1 2 3 4

20. CULTURAL CONTEXT OF SCHOOL AND COMMUNITY

Student teacher appears to be unaware of the cultural context of the school and community.

Student teacher demonstrates knowledge of the cultural context of the school and the community.

Student teacher demonstrates sufficient knowledge of the cultural context of the school and the community.

Student teacher demonstrates an expanding knowledge of the cultural context of the school and the community.

1 2 3 4

21. ABILITY TO REFLECT

Student teacher has no suggestions for how a lesson may be improved another time.

Student teacher makes general suggestions about how a lesson may be improved.

Student teacher is becoming a reflective practitioner and makes a few specific suggestions of what might be tried if the lesson was taught again.

Student teacher is a reflective practitioner, is able to learn from mistakes and successes and adjusts accordingly.

1 2 3 4


159

OTHER COMMENTS PLANNING AND PREPARATION

CLASSROOM ENVIRONMENT

INSTRUCTION

PROFESSIONAL RESPONSIBILITIES


160

CENTER FOR RESEARCH ON TEACHING AND LEARNING

Educational Beliefs and Multicultural Attitudes Survey TODAY’S DATE: NET ID: STUDENT ID NUMBER (N#): In an ongoing effort to get your feedback to help inform improvements in our teacher education programs at Steinhardt, we are asking you to complete this survey about your beliefs and attitudes about teaching, the role of culture in education, and your efficacy as a prospective teacher. Completion of this survey is voluntary and will not affect your standing in the program. Your responses are confidential and will be used only for program development and evaluation. Thank you for your participation! 1. Degree Level (Check one): Undergraduate Graduate*

*If you are a graduate student, are you in a fast track program? YES NO

2. Certification Program: (Check all that apply):

Art Education Literacy Education Childhood Ed Mathematics Education Childhood Ed/Childhood Special Ed Music Education Early Childhood Ed TESOL/Bilingual Ed Early Childhood Ed/Early Childhood Special Ed Special Education Educational Theatre Speech-Language Pathology and Audiology English Education Social Studies Education Dance Education Science Education Foreign Language Education Other:

3. Gender: Male Female

4. Are you an international student? Yes No

5. What language(s) do you usually speak at home? Only a language other than English More another language than English English and another language equally More English than another language Only English

6. Ethnicity: (e.g. Latino/a; African American; Asian; White/Euro-American )

7. Do you have any prior experience in teaching (e.g.: tutoring, student teaching, classroom teaching, etc)?

YES NO Please specify:

7a. Have you student-taught prior to this semester? YES* NO *If yes, how many times? 7b. Have you ever previously worked with minority students in a school setting? YES* NO

*If yes, what is your estimate of the percentage of minority students in that school setting?

8. Total number of credits completed in your program at NYU (DO NOT include current semester) Check one:

0-15 16-30 31-45 46-60 61-75 76-90 91-105 106 -120 121 or more


161

DIRECTIONS: Please indicate the extent to which you agree or disagree with each statement below by circling the appropriate numeral to the right of each statement.

1 STRONGLY DISAGREE

2 MODERATELY

DISAGREE

3 SLIGHTLY DISAGREE

4 SLIGHTLY

AGREE

5 MODERATELY

AGREE

6 STRONGLY

AGREE

1.To be an effective teacher, one needs to be aware of cultural differences present in the classroom.

1 2 3 4 5 6

2. I can learn a great deal from students with culturally different backgrounds.

1 2 3 4 5 6

3. Teachers are not a very powerful influence on student achievement when all factors are considered.

1 2 3 4 5 6

4. I have enough training to deal with almost any learning problem.

1 2 3 4 5 6

5. Being multiculturally aware is not relevant for the subject I teach.

1 2 3 4 5 6

6. The amount a student can learn is primarily related to family background.

1 2 3 4 5 6

7. Teachers have the responsibility to be aware of their students' cultural background.

1 2 3 4 5 6

8. If one of my students couldn't do a class assignment, I would be able to accurately assess whether the assignment was at the correct level of difficulty.

1 2 3 4 5 6

9. Teachers are responsible for developing their students’ sense of community.

1 2 3 4 5 6


162

10. When a student does better than usually, many times it is because I exert a little extra effort.

1 2 3 4 5 6

11. If a student in my class becomes disruptive and noisy, I feel assured that I know some techniques to redirect him/her quickly.

1 2 3 4 5 6

12. Teaching methods need to be adapted to meet the needs of a culturally diverse student group.

1 2 3 4 5 6

13. Multicultural training for teachers is not necessary.

1 2 3 4 5 6

14. When the grades of my students improve, it is usually because I found more effective teaching approaches.

1 2 3 4 5 6

15. The problems and realities that exist in low-income and low-performing schools are too complex for teachers to address.

1 2 3 4 5 6

16. Multicultural awareness training can help me to work more effectively with a diverse student population.

1 2 3 4 5 6

17. My teacher training program and/or experience will give (has given) me the necessary skills to be an effective teacher.

1 2 3 4 5 6

18. In all cases, respecting students is a teacher's obligation.

1 2 3 4 5 6


163

19. If a student masters a new concept quickly, this might be because I knew the necessary steps in teaching that concept.

1 2 3 4 5 6

20. The education system is obliged to address social injustice.

1 2 3 4 5 6

21. Most students with disabilities can and should be accommodated within general education classrooms.

1 2 3 4 5 6

22. When it comes down to it, a teacher can't do much because most of a student's motivation and performance depends on his or her home environment.

1 2 3 4 5 6

23. Teaching students about cultural diversity will only create conflict in the classroom.

1 2 3 4 5 6

24. When a student gets a better grade than he/she usually gets, it is usually because I found better ways of teaching that student.

1 2 3 4 5 6

25. I think that there is too much emphasis placed on multicultural awareness and training for teachers.

1 2 3 4 5 6

26. It is the role of the teacher to model and instruct students in social skills (e.g., caring and cooperation).

1 2 3 4 5 6

27.If a student did not remember information I gave in a previous lesson, I would know how to increase his/her retention in the next lesson.

1 2 3 4 5 6

28. All students can be taught to think critically.

1 2 3 4 5 6


164

End of Term Feedback Questionnaire for Student Teachers (ETFQ)

School Evaluation

Hello, . This semester, you were placed at Please answer the following questions about to the best of your ability.

1. Your Grade Level Placement:

Pre-K K 1 2 3 4 5 6 7 8 9 10 11 12

2. How well maintained is the physical facility?

o Poorly o Adequately o Well o Very well

3. How would you rate the general climate of ? (e.g., tone, safety, friendliness and openness of staff and administration)

o Excellent o Good o Fair o Poor

Cooperating Teacher Evaluation

This semester, you were assigned. Please evaluate to the best of your ability.

4. Approximately how many years of teaching experience does have?

First year teacher 2 to 5 years 6 to 10 years More than 10 years Don’t know

5. Please rate on the following items

Ver

y w

ell

Wel

l A

vera

ge

Poo

rly

Ver

y po

orly

How well did your cooperating teacher include you in the school community? (e.g. introduce you to the faculty, school resources and facilities)

How well did your cooperating teacher provide you with opportunities to take control of the classroom?


How would you rate the classroom environment established by the cooperating teacher? (e.g., safety, orderliness, respect, friendliness) How would you rate your cooperating teacher's assistance in furthering your organizational teaching skills? (e.g. planning, structuring lessons, assessment) How would you rate your cooperating teacher's assistance in helping you to enhance your teaching practice? (e.g.inform you about instructional philosophies and methods used in the classroom) How would you rate your cooperating teacher's assistance in helping you to develop content knowledge specific to your field and age group? (e.g. math, science, early childhood, etc.) How would you rate your cooperating teacher's assistance in helping you to become a reflective practitioner? (e.g., observing your teaching, confering with you on a regular basis) How would you rate your cooperating teacher's assistance in developing your classroom management skills? (e.g., routines, norms, student discipline) How would you rate your cooperating teacher's availability? How would you rate the quality of your rapport with your cooperating teacher? In summary, rate the cooperating teacher's overall assistance

Excellent Good Avera

ge

Fair

P

oor

How would you rate your supervisor's assistance in furthering your organizational teaching skills? (e.g. planning, structuring lessons, assessment methods) How would you rate your supervisor's assistance in helping you to enhance your teaching practice? (e.g. inform you about instructional philosophies and methods used in the classroom) How would you rate your supervisor's assistance in helping you to develop content knowledge specific to your field or age group you work with? (e.g. math, science, early childhood, etc.) How would you rate your supervisor's assistance in helping you to become a reflective practitioner? (e.g., observing your teaching, confering with you on a regular basis) How would you rate your supervisor's assistance in developing your classroom management skills? (e.g., routines, norms, student discipline) How would you rate your supervisor's availability? How would you rate the quality of your rapport with your supervisor?

Excellent Good Avera

ge

Fair

P

oor Ex

celle

nt

Exce

llent

Goo

d G

ood

Aver

age

Aver

age

Fair

Fair

Poo

r P

oor

6. Please rate on the following items

End of Term Feedback Questionnaire for Student Teachers (ETFQ)

How would you rate the classroom environment established by the cooperating teacher? (e.g., safety, orderliness, respect, friendliness) How would you rate your cooperating teacher's assistance in furthering your organizational teaching skills? (e.g. planning, structuring lessons, assessment) How would you rate your cooperating teacher's assistance in helping you to enhance your teaching practice? (e.g.inform you about instructional philosophies and methods used in the classroom) How would you rate your cooperating teacher's assistance in helping you to develop content knowledge specific to your field and age group? (e.g. math, science, early childhood, etc.) How would you rate your cooperating teacher's assistance in helping you to become a reflective practitioner? (e.g., observing your teaching, conferring with you on a regular basis) How would you rate your cooperating teacher's assistance in developing your classroom management skills? (e.g., routines, norms, student discipline) How would you rate your cooperating teacher's availability? How would you rate the quality of your rapport with your cooperating teacher? In summary, rate the cooperating teacher's overall assistance

7. Please identify and evaluate (as good, helpful, not helpful, etc.) specific ways in which responded to you and your teaching, e.g., ways in which support, advice and assistance were offered and resources shared. If the experience was not a good one, please describe the aspects of the cooperating teacher's behavior that seemed problematic: Supervisor Evaluation

This semester, you were assigned . Please evaluate to the best of your ability. 8. Please rate your Supervisor on the following criteria

How would you rate your supervisor's assistance in furthering your organizational teaching skills? (e.g. planning, structuring lessons, assessment methods) How would you rate your supervisor's assistance in helping you to enhance your teaching practice? (e.g. inform you about instructional philosophies and methods used in the classroom) How would you rate your supervisor's assistance in helping you to develop content knowledge specific to your field or age group you work with? (e.g. math, science, early childhood, etc.) How would you rate your supervisor's assistance in helping you to become a reflective practitioner? (e.g., observing your teaching, confering with you on a regular basis) How would you rate your supervisor's assistance in developing your classroom management skills? (e.g., routines, norms, student discipline) How would you rate your supervisor's availability? How would you rate the quality of your rapport with your supervisor?

9. Were you told the number of times that your supervisor would visit?

o Yes o No

10. If yes, how many times did you expect your supervisor to visit?

Once Twice Three Times Four Times Five Times More than Five Times

11. Did your supervisor visit as frequently as you were told s/he would?

More frequently than I was told As frequently as I was told Less frequently than I was told I was not told how often the supervisor would come


166

End of Term Feedback Questionnaire for Student Teachers (ETFQ) 12. In summary, rate the supervisor's overall assistance (check one)

o Excellent o Good o Fair o Poor

13. Please identify and evaluate (as good, helpful, not helpful, etc.) specific strategies which your supervisor may have used with you, e.g., informal conversations, email discussions, observations, conferences etc. If the experience was not a good one, please describe the aspects of the supervisor's behavior that seemed problematic.


167

Utilize different pedagogical approaches Have a mastery of your subject area Maintain order and discipline in the classroom Understand how students learn Impact your students ability to learn Implement state or district curriculum and performance standards Use student performance assessment techniques Address needs of students with disabilities Address needs of students with limited English proficiency Address needs of students from diverse cultural backgrounds Work with parents Work collaboratively with teachers, administrators, and other school personnel Identify and utilize the resources within the community where you teach Participate as a stakeholder in the community where you teach Integrate Technology into teaching

VeryWell Moderately

Well So

mewha

tWell

All Not V

ery

Wel

l M

oder

ately

W

ell

Som

ewha

t W

ell

Not

Wel

l at

All

Integrate Technology into teaching

Program Exit Survey 2010

Page One

1. Completion of the survey makes you eligible for optional entry into a lottery for an 8 Gig iPod nano digital player. If you would like to join in the lottery, please provide your email address:

2. NYU NetID (Your initials followed by a number):

3. Degree:

Bachelor of Science Master of Arts Advanced Certificate Bachelor of Music

4. Program area:

Art Education/Therapy/Studio Bilingual Education Childhood Education Dual Childhood Education/Childhood Special Education Dance Education Early Childhood Education Dual Early Childhood Education/Early Childhood Special Education Educational Theatre English Education Mathematics Education Music Education Science Education Social Studies Special Education TESOL Other (please specify)

Untitled Page

5. In your opinion, how well did your teacher preparation program at NYU prepare you with the following skills and knowledge to begin teaching

Utilize different pedagogical approaches Have a mastery of your subject area Maintain order and discipline in the classroom Understand how students learn Impact your students ability to learn Implement state or district curriculum and performance standards Use student performance assessment techniques Address needs of students with disabilities Address needs of students with limited English proficiency Address needs of students from diverse cultural backgrounds Work with parents Work collaboratively with teachers, administrators, and other school personnel Identify and utilize the resources within the community where you teach Participate as a stakeholder in the community where you teach


168

Subject matter knowledge Pedagogical knowledge Ability to manage the realities of teaching in public schools

Veryeffective Somewhateffective Somewhatineff

ective

Veryinef

fective

Subject matter knowledge Veryeffective Somewhateffective Somewhatineffective Veryineffective Pedagogical knowledge Ability to manage the realities of teaching in public schools

Ver

y ef

fect

ive

Ver

y ef

fect

ive

Som

ewha

t ef

fect

ive

Som

ewha

t ef

fect

ive

Som

ewha

t in

effe

ctive

So

mew

hat

inef

fect

ive

Ver

y in

effe

ctive

V

ery

inef

fect

ive

6. What are your plans for the next school year? (Required)

o Attend graduate school full time o Teach in a NYC public school o Teach in a public school outside NYC o Teach in a private school in NYC o Teach in a private school outside NYC o Work in a non-teaching job o I do not have plans yet o Other (please specify):

7. What would be your purpose in pursuing further education? (Please check all that apply) □ Deepen content knowledge □ Deepen pedagogical knowledge □ Qualify for salary increase □ Obtain another degree □ Obtain additional certification □ Qualify for promotion □ Train for a career other than teaching □ Other (please specify)

8. Which school and district/borough (if known)?

9. What will be your job title (if known)?

10. Which career will you be trained for?

11. Which school and borough?

12. Which school and location?

Untitled Page


13. Please rate the overall effectiveness of fieldwork observations (100 hours) in advancing your learning for each of the following.


14. Please rate the overall effectiveness of student teaching in advancing your learning for each of the following.



169


Veryeffective Somewhateffective Somewhatineff

ective

Veryinef

fective

Registration process Excellent Good Fair Poor

Advisement Program schedule Physical facilities Opportunities for interaction with your cohort Opportunities for interaction with faculty

Ver

y ef

fect

ive

Exce

llent

So

mew

hat

effe

ctiv

e G

ood

Som

ewha

t in

effe

ctive

Fa

ir

Ver

y in

effe

ctive

P

oor


15. Please rate the overall effectiveness of Steinhardt course work in advancing your learning for each of the following.


16. Which Steinhardt courses/experiences best prepared you with the skills and knowledge to begin teaching?

17. Which Steinhardt courses/experiences least prepared you with the skills and knowledge to begin teaching?

18. Please rate the overall quality of Steinhardt faculty.

□ Excellent □ Good □ Fair □ Poor

19. Please rate the overall level of coherence between method courses and field experiences. o Excellent o Good o Fair o Poor

Untitled Page

20. Please rate the following logistical arrangements as they contributed to your success in the program.

Registration process Advisement Program schedule Physical facilities Opportunities for interaction with your cohort Opportunities for interaction with faculty

21. What is your overall rating of the program? □ Excellent □ Good □ Fair □ Poor

22. Please share any of your other comments related to your study at NYU. 23. We would like to stay in contact with you as you pursue your career beyond graduation. Please indicate the ema1l address we may use to contact you:


170

First Year Teacher: Feedback Form

Page One

1. Completion of the survey makes you eligible for optional entry into a lottery for an 8 Gig iPod nano digital player. There will be three winners in total. If you would like to join in the lottery, please provide your email address:</span

2. Graduation Date o January 2009 o May 2009

3. The degree that you earned at NYU o BS o MA o Advanced Certificate

4. Your certification area (s):

Art Education/Therapy/Studio Bilingual Education Childhood Education Dual Childhood Education/Childhood Special Education Dance Education Early Childhood Education Dual Early Childhood Education/Early Childhood Special Education Educational Theatre English Education Mathematics Education Music Education Science Education Social Studies Special Education TESOL Other (please specify)

Untitled Page

5. Current teaching status (Required)

o Teaching in a NYC public school o Teaching in a NYC private school o Teaching in a school outside of NYC o Not teaching at the present time

6. What school and district are you teaching in?

7. What grade level are you currently teaching?

8. What subject are you currently teaching?

9. Are you currently o Attending graduate school full time? o Working full time


171

o Other (please specify) 10. What is the name of your graduate school? What degree and program are you in?

11. Where are you working and what is your position?

12. Do you plan on entering teaching? o Yes o No

13. Where do you plan on entering teaching? o New York City public school o New York City private school o outside of New York City o Not sure

14. Why do you not plan on entering teaching?

15. Which best describes where you see yourself teaching in the next 2-5 years

Continue teaching in the same school Teach in a different school in the same New York City district Teach at different New York City district Teach outside of New York City Leave teaching

16. Which best describes where you see yourself teaching in the next 2-5 years?

Continue teaching in the same school Teach in a New York City public school Teach in a school outside of New York City Leave teaching

17. Which best describes where you see yourself teaching in the next 2-5 years?

Continue teaching in the same school Teach in a New York City school Teach in another school outside of New York City Leave teaching


172

Utilize different pedagogical approaches Have a mastery of your subject area Maintain order and discipline in the classroom Understand how students learn Impact your students ability to learn Implement state or district curriculum and performance standards Use student performance assessment techniques Address needs of students with disabilities Address needs of students with limited English proficiency Address needs of students from diverse cultural backgrounds Work with parents Work collaboratively with teachers, administrators, and other school personnel Identify and utilize the resources within the community where you teach Participate as a stakeholder in the community where you teach Integrate technology into the grade level or subject taught

VeryWell Moderately

Well So

mewha

tWell

All Not

Core courses Methods courses Curriculum/content courses Other NYU courses LStueadrneinntgTPaeartnchienrgexperience (pre student teaching)

ExtremelyUseful SomewhatUs

eful

Notuse

ful

All

Ext

rem

ely

Use

ful

Ver

y W

ell

Mod

erat

ely

Wel

l So

mew

hat

Wel

l N

ot W

ell

at A

ll

Som

ewha

t U

sefu

l

Not

ver

y us

eful

Not

Use

ful

at A

ll

18. In your opinion, how well did your teacher preparation program at NYU prepare you to:

Utilize different pedagogical approaches Have a mastery of your subject area Maintain order and discipline in the classroom Understand how students learn Impact your students ability to learn Implement state or district curriculum and performance standards Use student performance assessment techniques Address needs of students with disabilities Address needs of students with limited English proficiency Address needs of students from diverse cultural backgrounds Work with parents Work collaboratively with teachers, administrators, and other school personnel Identify and utilize the resources within the community where you teach Participate as a stakeholder in the community where you teach Integrate technology into the grade level or subject taught

Untitled Page

19. How valuable were the following parts of your NYU experience in preparing you for teaching:

Core courses Methods courses Curriculum/content courses Other NYU courses Learning Partner experience (pre student teaching) Student Teaching

20. What course/experience at NYU do you feel was most helpful in preparing you for teaching?

21. What course/experience at NYU was least helpful in preparing you for teaching?

22. In which aspect(s) of teaching did NYU prepare you best?


173

23. In which aspect(s) of teaching did NYU prepare you least?

24. What recommendations do you have for NYU to improve its teacher education programs for future teachers?

25. Are you receiving assistance from a mentor teacher? o Yes o No

26. How often do you see the mentor? o twice weekly o once weekly o every ten days o rarely

27. What does the mentor do with you? o observes my lessons and offers feedback o provides teaching resources o helps me with lesson plans o invites me to observe his/her teaching o helps me review assessments and student work o offers general advice and information o other

28. How effective is the help from the mentor? o Very effective o Somewhat effective o Somewhat ineffective o Very ineffective

29. When you work with the mentor, do you discuss any form of teaching standards? o Yes o No o Not Sure

30. Would you be willing to participate in a focus group? (For more information, please contact [email protected])

o Yes o No

31. NYU and the DOE are offering additional mentoring support to complement the support you are receiving at your school. Would you be interested in receiving such support? (For more information, please contact [email protected] )

o Yes o No


174

APPENDIX G: Accreditation of NYU Professional Education Programs Additional Program Titles listed in the New York State Education Department Inventory of Registered Programs (NYSED IRP), with the Accreditation Areas and Accreditation Agency are listed in Table G.1 TABLE G.1 Accreditation of NYU Professional Education Programs

Accreditation Area: Accreditation Agency (submitted by NYU)

Code (NYSED IRP)

Program Title (NYSED IRP)

HEGIS (NYSED IRP)

Award (NYSED IRP)

Notation:

Professional Child/School Psychology (PsyD): American Psychological Association

79311 CHILD/SCHOOL PSYCHOLOGY 2099 PSY D

On moratorium-‐being phased out

NYU status: accredited-‐but inactive: http://www.apa.org/ed/accreditation/programs/accred-‐

school.aspx

Speech-‐Language Pathology (MA): American Speech-‐Language and Hearing Association-‐Council on Academic Accreditation

24691 COMMUNICATIVE SCIENCES AND DISORDERS

1220 MS

See letter p. 176

Counseling Psychology (PhD): American Psychological Association

13853 COUNSELING AND GUIDANCE 0826.01 PHD

See letter p. 175

School Psychology (PhD and PsyD): American Psychological Association

7890 SCHOOL PSYCHOLOGY 2099 PHD

PsyD

On moratorium-‐being phased out

NYU status: accredited-‐but inactive: http://www.apa.org/ed/accreditation/programs/accred-‐

school.aspx Letters from the American Psychological Association for the Counseling Psychology Ph.D, and the Council on Academic Accreditation in Audiology and Speech Language Pathology for Communicative Sciences and Disorders (MS) are provided below:


175

OCT-12-2011 10:40 p.001/001

AMERICAN PSYCHOLOGICAL ASSOCIATION

August 19, 2008

John E. Sexton, Ph.D., J.D. President Office of the President New York University New York, NY 10003

Dear President Sexton,

At its meeting on July 17-20, 2008, the Commission on Accreditation (formerly the Committee on Accreditation) conducted a review of the doctoral Ph.D. program in Counseling Psychology at New York University. This review included consideration of the program's most recent self study report, the preliminary review of October 12, 2007 and the program's response to the preliminary review on February 27, 2008, the report of the team that visited the program on April 17-18, 2008, and the program's response to the site visit report on June 12,2008.

I am pleased to inform you that, on the basis of this review, the Commission voted to award accreditation to this program. In so doing, the Commission scheduled the next accreditation site

-. visit to be held in 2013. During the interim, the program will be listed annually among accredited programs of professional psychology in the American Psychologist and on the Accreditation web pages. The Commission also encourages you to share information about your program's accredited status with agencies and others of the public as appropriate.

Drs. Carlton Parks, Ruperto Perez, and Norma Simon recused and therefore did not participate in the discussion and vote on your program.]

The Commission would like to provide the program with a summary of its perceived relative strengths and weaknesses. 1bis will be provided below according to each of tl(e accreditation domains. At the end of the letter, the program will be provided with an itemized list of any actions that the program needs to take prior to the next accreditation review._ A summary of the Commission's review of this program is provided below.

Domain A: Eligibility As a prerequisite for accreditation, the program's purpose must be within the scope of the accrediting body and must be pursued in an institutional setting appropriate for the doctoral education and training of professional psychologists.

750 First Street. NE Wosh;ngton, DC 20002·∙4242 12021 336·∙5500 12021 336-‐6123 TDD


176

OCT-12-2011 10:39 P.001

CERTIFIED RETURN RECEIPT

August 28, 2009

Celia F. Stewart, PhD New York University Steinhardt School of Education Dept. of Speech-Language Pathology & Audiology 719 Broadway, Suite 200 New York, NY 10003

CAA File #15 - Master's program in speech-language pathology

Dear Dr. Stewart,

I am pleased to inform you that during its meeting on July 29 - August 1, 2009, the Council on Academic Accreditation in Audiology and Speech-Language Pathology (CAA) voted to re-accredit the graduate education program in speech-language pathology at New York University for a period of 8 years beginning July 1, 2009 through June 30,2017.

The issues provided in the attached Accreditation Action Report should be addressed in the program's next annual report. Your first annual report will be submitted using the on Iiue report format via the Higher Education System and according to the revised Annual Report submission date on August 1, 2010. Approximately three months prior to the due date of your reports, you will be sent a reminder .that the program's next report to the CAA is due for submission.

Notification of Program Changes: In accordance with Standard 1.5 and 1.7, notification of any change to the program director must be provided in writing to the CAA within 30 days of the change. This notification should include reporting temporary or interim replacements resulting from searches for a new program director and sabbatical leave. Notice of a change should also include a vita for the new/interim director and the program's plan for implementation of the change.

2200 Research Boulevard, Mail Stop 310 Rockville, MD 20850·3289


177

OCT-12-2011 10:39 P.002

Congratulations to you, the faculty and staff in the program, as well as the administration, on this national distinction.

Sincerely,

.o- .....PT

Judith L. Page, PhD, Chair Council on Academic Accreditation in Audiology and Speech-Language Pathology

cc: Barnett W. Hamberger, Associate Provost for Academic Program Review

Susan Flesher, ASHA National Office CAA Members

New York University Teacher Education Program TEAC Inquiry ...€¦ · 4.2.5c Mean CCT GPA by...

Documents

Transcript of New York University Teacher Education Program TEAC Inquiry ...€¦ · 4.2.5c Mean CCT GPA by...