1

CollegeorUnitLevelAnnualAssessmentReport

TemplateandGuidelines(Rev.May30,2016)

CollegeorUnitName:CollegeofEducationReportYear: 2015-16Submittedby:BethKubitskey,AssociateDeanoftheCollegeofEducationSubmittedon(date):DUEJUNE30

EMU’sMissionandExpectationforAssessment

(https://www.emich.edu/assessment/)MissionEMUcreatesacultureofassessmentthroughcollaborativeplanning,systematicimplementation,andrigorousanalysisofcollecteddatatomakeinformeddecisionsthatenhanceopportunitiesforstudentstolearnandtostrengthenallcurricularandco-curricularareas.ExpectationEMUexpectsallcurricularandco-curricularareastogenerateandimplementlearninggoals,collectrelevantdata,anduseon-goingassessmentprocessesforcontinuousimprovement.PurposeofUnitReportsonAssessmentofStudentLearningThenineunitsthatreportonassessmentofstudentlearning(seethelistbelow),listtheirgoalsfortheacademicyear,describewhatgoalswereaccomplished,andprovideexamplesofhowassessmentdatawereusedtoenhanceprograms.NoteonPreparationforPreliminaryVisitEMUispreparingforapreliminary(“mock”)HigherLearningCommissionvisit(scheduledforNovember10,2016);therefore,theinformationyouprovidemaybeusefultotheHLCPlanningTeams,particularlyteams#3(TeachingandLearning:Quality,Resources,andSupport)and#4(TeachingandLearning:EvaluationandSupport).Forlinkstotheassessmentpageforeachofthefollowing,gotohttps://www.emich.edu/assessment/unitsaaessment.php

• CollegeofArtsandSciences• CollegeofBusiness• CollegeofEducation• CollegeofHealthandHumanServices• CollegeofTechnology• GeneralEducation• GraduateSchool• StudentAffairs&StudentServices• UniversityLibraryPleaseaddressallsevenitems.

2

1. DescriptionofCouncil/Committee.Describehowyourassessmentcouncilorcommitteeisorganizedandprovidealistofthefacultyandstaffwhodirectlycontributetoit.

TheAssessmentCommitteeintheCollegeofEducationisasubcommitteeoftheCollegeofEducationFacultyCouncil.ItincludesarepresentativefromeachoftheCOEdepartmentsaswellasamemberfromoutsidethecollege,butpartoftheinitialteacherpreparationprogram.Inaddition,theCOEAssociateDeanofStudentandCurriculumparticipates.Thecommitteemeetsmonthlytodiscussissuesofcollegeassessment.Thisincludes,butisnotlimitedto,datacollectionforaccreditingpurposes.

o Dr.MarthaBaiyee TeacherEducationo ProfessorJenniferDesiderio SpecialEducationo Dr.DavidAnderson LeadershipandCounselingo Dr.CamMcComb Chair,CASMethodsGroup(Art)o Dr.BethKubitskey AssociateDean,COE

2. AssessmentGoals.Inadditiontotheprimarygoalofassessingstudentlearning,listother2015-

16unitgoalsthatweretosupportassessmentofstudentlearning(notewhetherthesearedirect,indirectoroperational).Goal1. Improvestudentteachingexperienceandassessment.

a. Adaptcapstoneassessmentforinitialteacherpreparationprograms.(direct)b. Train student teacher supervisors in the new assignment and scoring. (operational)

Goal2. Recreatetheprogramreviewprocessfornon-accreditedprograms.(operational)

a. EducationLeadershipsharingprotocolfortheirprogramreviewprocess.b. Inprogress.

Goal3. Improvesupportforfacultytomakedatadrivendecisionsabouttheirprogramin

annualprogramevaluations.a. Createdlivetextdataportalforeachprogramthatincludesstudentlearningdata

(rubricsfromcourseassignments,MTTCtestdata,GPAs)(direct)aswellasprogramleveldata(enrollmentetc)(operational)andStatedata(indirect).

b. Createdprotocolforprogramstousethedata.(procedural)

Goal4. MaintainnationalrecognitionofprogrambysubmittingSpecializedProfessionalAssociations(SPA)reportsa. SpecialEducationPrograms(direct)b. EducationalTechnologyProgram(direct)1c. Reading(direct)d. (CASalsosubmittedreports–notincludedinthisreport)

Goal5. Createmechanismforcollectingdataoncompleters’preparednesstoinform

programs.a. WorkingwithotherinstitutionsacrossMichiganaboutimplementingsurveysimilarto

Ohio’s.(indirect).

1 Elementary program SPA is no longer involved with CAEP and review is transition to the CAEP board. Early childhood is submitting this year.

3

Goal6. Havevalidandreliableinstrumentsformeasuringcandidate(student)learning-

validatestudent-learningrubrics.a. edTPAtrainingandcreatingprotocolforinter-raterreliability.(operational)b. Specialeducationreworkedtheircourserubrics.(operational)c. Workedondispositionassessment.Continuingworkwithotherteacherpreparation

programsinMichigan.(operational)3. SummaryofAccomplishments.Summarizetheaccomplishmentsyourunitachievedduring

2015-16towardassessingstudentlearning(theprimarygoal).Next,summarizetheactivitiesyourunitengagedinduring2015-16towardmeetingothergoalslistedabove.

• AssessingStudentLearning:Accomplishment1-AdoptionofedTPA:Wemetgoal1byreplacingourstudentteachingunitwiththeedTPA(localevaluation).Thiswasamajortaskrequiringprofessionaldevelopmentforourstudentteachersupervisors.Thiswasthepilotyearandonlyincludedsecondarystudentteachers.Inthefalladoptionwedidnotallowsupervisorstoevaluatetheirownstudents.Insteadwedistributedtheportfoliotosupervisorswhowerecertifiedintheareaofthestudentteacher.Duringwinterweallowedsupervisorstoevaluatetheirownstudents.Wearepresentlycomparingthetwoapproachestodecideonnextsteps.Twomajorfindingstodate:

o TheELAportfolioaswrittendidnotreflectthebreadthofourprogram.OurELAcolleaguesre-wrotetherubrictobemoreflexiblewithrespecttothetypeoflessonandareutilizingthisrubric.

o WehavecomeupwithscaffoldedduedatesforthepartsoftheedTPAtohelpsupervisorsproviderichermorefocusedfeedbackinatimelymanner.

Accomplishment2-TwoactionitemsfromSPAreportfeedback:Feedbackfromgoal4suggestedareasforimprovementinourprograms/programreporting.Wemadethefollowingadaptionsinourprograms.(Note:InitialteacherpreparationprogramsinCASmadeoraremakingsimilaradaptations.)

o Re-examinedassessments:SpecialEducationisre-evaluatingtheirassessmentsandrubrics.Theyhavehadtwofacultyretreatsandmakingchangesbasedonevaluationoftheusefulnessofthefindingsandtherubrics.EducationalMediaandTechnology(advancedprograms)alsoadaptedtheirrubricstobetteralignwithstandards.

o Examinedreportingtiming:Reading(advancedprogram)wasgoingtobedoingalotofadaptationstomeetantiquatedstandards.ExaminingpostponingSPArecognitionforafewyearsbecause(a)haveanewprogramandwanttoevaluatethatprogramand(b)newstandardsareoutnowthatareincorporatedinthenewprogram,makingtheworktoevaluatetheoldprogramlessuseful.Ultimatedecidedtoresubmit.

• MeetingAssessmentGoals:

o Goal1-edTPA:Pleaseseeabove.o Goal2–Programreview:ModelbeingworkedoninLeadershipandCounseling.o Goal3–Dataportal:Dataportalsdesignedandbeingpopulatedforfallrelease(PADS,

AppendixB).o Goal4–SPAreports.SPAreportssubmitted.

§ Specialeducationisworkingonaddressingissuesbroughtupbyevaluation

4

§ EducationTechnologyhassubmittedrevisedreport.§ Reading–seeabove.§ (CASreportsalsosubmitted).

o Goal5–Employersatisfaction:§ Createdsamplesurvey.§ ReviewedbyPEAC(findings-toolong).§ IdentifiedOhio’ssurveyasbeingexemplarforCAEP.§ WorkingonusingOhio’ssurvey(andpossiblywithotherinstitutions).

o Goal6–Validationofrubrics-Inprogress.4. Examples.Provide2-3descriptiveexamplesfromyourunit’sactivitiesthathighlighthowyou

assessedstudentlearning,including“closingtheloop.”• Theexamplesmightbeonesthatindirectlyinfluencestudentlearning(e.g.,reorganizing

assessmentcouncils,revisingtemplates,etc.).• However,atleastoneoftheexamplesshoulddescribeadirectmeasureorapproachto

assessingstudentlearning(e.g.,anindividualprogram’sexample).

• Example1(direct/indirect):edTPA-WecompletedoursecondaryedTPAadoption.Thisprocesswasnotwithoutchallenges.Wetrainedsupervisors/instructorsusingaprotocoldesignedbytheedTPAcreators(SCALE).Onereasonweadoptedthisassignmentwasbecauseofitseducativevaluetothestudents.However,changeisalwaysachallenge.Inthefallsemesterweusedtheacceptedprotocolofassigninggraderswhowerenottheinstructorsforthestudents.Thisshufflingprovedchallenginginmatchingtheexpertise(endorsement)withteachingload.Inwintersemestersupervisorsexpressedaninterestinevaluatingtheirownstudentsandcreatingamodelofturninginportionsoverthecourseofthestudentteachingforfeedbackaswasdonewiththepreviouscurriculumunit.Supervisorsevaluatedtheirownstudents,andinFall17wewillpilotthestaggeredsubmissionofpartsmodel.Wearepresentlyanalyzingthedatatoidentifystrengthsandweaknessesintheprogram.(SeeAppendixA)

• Example2(indirect/procedural):ProgramAreaDocuments(PADs,AppendixB).Thecreationoftheonlineportaltoprogramleveldatawillmakeannualprogramevaluationeasierandmoreroutineforfaculty.IntheCOEwecollectanenormousamountofdata.Pastpracticehadthedataavailabletothefaculty,butinavarietyofplaces.Inaddition,theStateprovideswithavarietyoffeedbackfromsurveysthatwouldbeusefulataprogramlevel.PleaseseeattachmentforoutlineofdataincludedinthePAD.Inaddition,programswillbeexpectedtoreportontheirobservationsfromthefeedback(seeattached).

5. ClosingtheLoop.Discusswhatyourunitlearnedfromthe2015-16effortsofassessingstudent

learningandhowitwillusethefindingstoimprovetheprogram(s),unit,andopportunitiesforstudentstolearn.Inotherwords,howwillyourunitusefindingsto“closetheloop”andimprovetheprogram?• EdTPA:Wearejustanalyzingthedataforthecapstoneassessmentforteacherpreparation.

Theanalysisofthedatawillhelpusdeterminehowourcandidatesperformonnationallynormedassessments.Thesefindingswillhelpinformourprogramsandultimatelyfeedbackintothecoursedesigns.

• SpecialEducationSPAresponse:Specialeducationfacultyrealizedtheywerenotgettingtheinformationfromtheexistingrubricsthatwouldallowthemtosufficientlyevaluate

5

theirprograms.TheSpecialEducationDepartmenthostedatwodayeventtohavefacultyworkonupdatingtheirrubricstobettermeasurestudentlearningoutcomesthatalignwithnationalandlocalstandards.Thesenewrubricswillbeadoptedthisfollowingfall.

• PADS(addressedabove,AppendixB),willallowforanannualdatadrivenprogramevaluation.

6. NextYear’sGoals.Asyouturntowardthenextacademicyear(2016-17),listandbrieflydescribegoalsthatemergedfromthecurrentyearandthatyouwillfocusonnextyear?1. Employersatisfactiondatacomplete.2. Alignassessmentsincurriculum,reading,practicumandtechnologywithedTPA.3. Haveprogramreviewsystemsinplacefornon-accreditedprograms.4. Implementannualreportingtoolforprogramsusingdatafromdataportal.5. Collectdataoncompletersimpactonstudentlearning.6. Workonsupportingnon-teacherprepprogramsintheirprogramevaluationprocess.

7. ProvideTemplateusedforReporting.Finally,pleaseprovideacopyofarepresentative

templatethatyouusedforprogramstoreporttheirassessmentfindings.(SeeAppendixC.)

ReferenceLiterature:RoleofAssessmentofStudentLearningintheAccreditationProcessInorderforEMUtoearninstitutionalaccreditation,TheHigherLearningCommissionexpectstheuniversitytomeetfivecriteria(http://policy.ncahlc.org/Policies/criteria-for-accreditation.html),andCriterion4focusesonassessmentofstudentlearning:

CriterionFour.TeachingandLearning:EvaluationandImprovementTheinstitutiondemonstratesresponsibilityforthequalityofitseducationalprograms,learningenvironments,andsupportservices,anditevaluatestheireffectivenessforstudentlearningthroughprocessesdesignedtopromotecontinuousimprovement.

TheHigherLearningCommissiondescribesthevalueofassessingstudentlearningasincludingthefollowing:“Forstudentlearning,acommitmenttoassessmentwouldmeanassessmentattheprogramlevelthatproceedsfromcleargoals,involvesfacultyatallpointsintheprocess,andanalyzestheassessmentresults;itwouldalsomeanthattheinstitutionimprovesitsprogramsorancillaryservicesorotheroperationsonthebasisofthoseanalyses.Institutionscommittedtoimprovementreviewtheirprogramsregularlyandseekexternaljudgment,advice,orbenchmarksintheirassessments.”(TheCriteriaforAccreditation:GuidingValues,http://www.ncahlc.org/Information-for-Institutions/guiding-values-new-criteria-for-accreditation.html.)

6

AppendixA.SampleedTPAdata(N=5)

2.672.83

2.672.83

2.67 2.602.80

2.402.60

2.40 2.502.67

2.502.33

2.67

Winter2016Science

7

Appendix B. PADS Sample.

PAD17a -Mathematics(SecondaryEducation)byEasternMichiganUniversityCollegeofEducation

AssessmentPlanMathematics-INITIAL<SPAreport><paragraphaboutaccreditation>Hereisalinktoyourstandards: http://www.nctm.org/Standards-and-Positions/CAEP-Standards/Pleasechecktomakesurethatyourrubricsareuptodate.

Reportson Assessments-LiveTextAssignmentsMath (Secondary)AssessmentsMath(Secondary)LivetextAssessments2013-16Pleaseclick,MathAssessments,andchoosetheyearsandcoursetoseethereports.GPAsGPAResultsPleasefindtheattachedfilewithavg.contentGPAsforcompleters. Additionalinformationavailableuponrequest. AttachmentsGPA_Template.xlsx

8

ProfessionalBehaviors(Dispositions)ProfessionalBehaviors2011-2016ToseetheresultscollectedthroughLiveText,click belowandthenclickonthesemesterwanted. Asemesterwillonlyappearifdatawascollected.ProfessionalBehaviorResultsCertificationTest ScoresMichiganTest for Teacher CertificationSummary of InstitutionResults DescriptiveInformationandInterpretiveCautionDetailsforMichiganTestforTeacherCertificationDataTableTheattachedtablespresentMichiganTestforTeacherCertification(MTTC)passingpercentagesbyinstitutionandbytestforcandidateswhotestedforthefirsttime.Thetablesincludefirst-timecandidateswhoweredeemedeligiblebytheinstitutionverificationprocess.Thecumulativepassingpercentagesincludepassingattemptsbythesecandidatesatanysubsequentadministration.Assuch,thecumulativepassingpercentageisthe“bestattempt”passingpercentageforaneligibletesttakerduringthe3-yearinterval.InterpretiveNotesandCautionsResultsreportedforonlyasmallnumberofcandidatesmaynotbeindicativeofhowlargenumbersofcandidatestypicallyperform.Candidateswhosedataarepresentedinthisdocumentmaynotreflectthesameperformanceasthatofcandidateswhowilltakethesetestsinthefuture.AttachmentsMTTCSubjectTest #22(Mathematics-Secondary)AtthebottomofthissectionisthelistofsubteststhatmakeuptheMathematics-Secondarysubjectareatestfor

9

MTTC.Thescorereportsonthesepagesincludedatafromstudentswhoareeligibletotakethesubjecttestatthetimeofthetestadministration.Assuch,thereportisasnapshot.Pleaserememberthatonlythosecandidateswhopasstheappropriatesubjecttestandcompleteallotherrequirementsareconsideredprogramcompleters.Programcompletersarerecommendedforstatelicensure.MichiganTest for TeacherCertificationSubjectAreaBreakdownTest 22Mathematics(Secondary):Subareas(Approx.%ofquestions):

1. MathematicsProcessandNumberConcepts(22%)2. Patterns,AlgebraicRel.,andFunctions(28%)3. Measurement andGeometry (22%)4. DataAnalysis,Stats.,Probability,andDisc.Math(28%)

MTTCResults022_2014_15_Major_MTTC_Subarea_breakdown.xls,022_2014_15_Minor_MTTC_Subarea_breakdown.xls,022_2013_14_Major_MTTC_Subarea_breakdown.xls,022_2013_14_Minor_MTTC_Subarea_breakdown.xls,022_2012_13_Major_MTTC_Subarea_breakdown.xls,022_2012_13_Minor_MTTC_Subarea_breakdown.xls,022_2011_12_Major_MTTC_Subarea_breakdown.xls,022_2011_12_Minor_MTTC_Subarea_breakdown.xlsMDEData

• StudentTeacher/CompleterExitSurveys• SupervisorSurveysCooperating• Teacher SurveysOneYearOut Surveys• Teacher Evaluations

EMUCollectedData

• PrincipalSurvey• Admission/Enrollment/GraduationData

10

Reviewingthe Data(seeAppendixC–Facultyreviewofcandidatedata)ProgramReportAIMSReportBelowisthereportfortheSecondaryMathematicsprogramatEasternMichiganUniversity.ThisSPAreportwassubmittedinfall2015toNCTMforreview.Attachments MathSecInst_5323_2_.pdfNationalRecognitionReportAttachedistheNational RecognitionReport.Attachments sec_math_NCTM_NRR.pdfCatalogCourse &ProgramInformationSecondary EducationMathematicsMajorChickHere:toviewtheEMUCataloginformationaboutthisprogram.Thelinkwillopeninanewwindoworanewtab,dependinguponhowyourbrowserisconfigured.Course &ProgramInformationSecondary EducationMathematicsMinorChickHere:toviewtheEMUCataloginformationaboutthisprogram.Thelinkwillopeninanewwindoworanewtab,dependinguponhowyourbrowserisconfigured.

Appendix C. Annual Program Examination Template (will be LiveText document)

1. Program Name: 2. Date(s) program met to discuss data:

From the list below, report your observations from the data provided in the PADS report.

3. Observations about LiveText Assessments: a. Are your rubrics up to date with standards? If not, do you have a plan? b. Are the rubrics sufficient? If not, do you have a plan? c. Other observations? d. N/A

4. Observations about grade point averages: a. How are students performing? b. Are there any concerns about meeting the cohort average requirement of 3.0? c. Other observations? d. N/A

5. Observations about dispositions: a. Is there anything missing that you think might be important? If so, what? b. Other observations? c. N/A

6. Observations about MTTC Test Scores: a. Strengths? b. Weaknesses? c. Other observations? d. N/A

7. Observations about MDE information: a. Observations about candidates’ exit survey information. b. Observations about supervisors’ survey. c. Observation about one year out survey. d. Observation about teacher evaluation. e. Other observations? f. N/A

8. Observation about admission/enrollment information. a. Marketing ideas? b. Impact on course offerings? c. Other observations? d. N/A

9. Based on this information, what goal would you like to work on this year? 10. What is your plan for working on this goal?

Follow-up on goal(s) 1. Program Name: 2. Date(s) program met to discuss status of goal: 3. Results: 4. Next steps (if any):