Weber State University Annual Assessment of Evidence …. Report of assessment results for the most...

1

Weber State University Annual Assessment of Evidence of Learning

Cover Page Department/Program: Master of Education in Curriculum and Assessment Academic Year of Report: 2013-2014 Date Submitted: December 19, 2014 Report author: Peggy J. Saunders, Ph.D., Program Director Contact Information: Phone: 801-626-7673 Email: [email protected]

2

A. Brief Introductory Statement:

New Statement – Spring Semester, 2014

The Master of Education (MED) Program is the oldest master’s program on the WSU campus. It began in 1978 as collaboration

between WSU and Utah State University; although, all the courses were taught by WSU Teacher Education faculty. In 1988, it

became the first stand-alone master’s degree on campus, three years before Weber became a university. The program caters to

practicing teachers, on-campus personnel wishing to pursue a master’s degree, and to people in business and medical fields who

teach as part of their jobs.

In an effort to help those who already had bachelor’s degrees and wanted to teach, the department added a licensure track for

secondary education in 2007 at the post-baccalaureate level. As long as a person had a degree in a subject taught in Utah schools,

he or she could successfully complete the coursework including student teaching and qualify for a Level 1 Utah Teaching License.

This part of the program was immediately successful. The following year, due to popular demand, an elementary licensing track

and a special education (mild/moderate) licensing track began. It is estimated that approximately two-thirds of the students

admitted each semester (usually between 22 and 30) are seeking a teaching license. The post-baccalaureate licensing program will be aligning its outcomes to the Utah Effective Teaching Standards. This portion of

the M.Ed. program is in the process of accreditation by the Teacher Education Accreditation Council. Many of the new

assessment tools are under development and will be piloted in spring, 2015.

B. Mission Statement Updated – Spring Semester, 2014

The mission of the Master of Education in Curriculum and Instruction (MEd) program is to extend the professional knowledge, skills,

and attitudes of educators in schools, business, industry, and higher education through advancing the theoretical and practical

applications of curriculum and instruction.

The program has a secondary mission of preparing post-baccalaureate students for an entry level teaching license in elementary

education, secondary education, or special education. The mission of this segment of the program aligns with the Teacher Education

Department’s mission: We work within our communities to prepare caring, competent educators and to promote equitable, inclusive,

and transformative education practices.

3

C. Student Learning Outcomes

Outcomes are current except for the dispositions information. It has been found over the past seven years of using the dispositions

form that professors have concerns about only a few of the master’s students. The faculty decided to use the form only in referral

situations rather than an on-going assessment tool because it yields limited data.

The post-baccalaureate licensing students’ dispositions will be assessed based on the Utah Effective Teaching Standards –

Standard 10 which reads: The teacher demonstrates the highest standard of legal, moral, and ethical conduct as specified in Utah

State Board Rule R277-515. The assessment tool will be piloted in spring semester, 2015.

D. Curriculum

Curriculum grid is current. E. Assessment Plan

Assessment plan is current with the exception of the dispositions as noted above.

4

F. Report of assessment results for the most previous academic year:

Evidence of MED Candidate Learning (Summer, 2013, Fall, 2013, & Spring, 2014)

Course Outcome Assessment Assessment Tool Data for 2013-2014 Academic Year

Candidates will demonstrate

MED 6000: Fundamentals of Graduate Study

Ability to analyze and critique educational research.

Preliminary Literature

Review

Rubric: Literature Review

for MED 6000

Instructor did not supply data.

MED 6010: Advanced Historical Foundations

Knowledge of the history and philosophy of

education. Group Presentation on

Ed. Foundations Topic

Rubric: Presentation MED

6010

The mean score is 25/25. A total of 39 students were in the

course during these three semesters.

MED 6020: Diversity in Education

Knowledge of issues related to differences among

groups of people and individuals and the impact on

teaching and learning.

Social Justice Action

Research Paper

Instructor grading No data available because professor no longer teaches at WSU.

Research Presentation

on an Assigned

Diversity Topic

Instructor grading

MED 6030: Advanced Educational Psychology

Knowledge of important theories of learning and

development and the implications for education. Oral Presentation of

Influential

Psychological Theorist

Rubric: Presentation MED

6030

The mean score is 48.97/50. The range is 44-50. A total of 38

students were in the course during fall & spring semesters.

MED 6050: Curriculum Design, Evaluation, & Assessment

Knowledge of principles of curriculum

development and assessment. UBD Curricular Unit

with Integration and

Differentiation

Rubric: Curricular Unit*

Mastery Model

The mean score is 70/70. A total of 44 students were in the

course during these three semesters.

MED 6060: Instructional Strategies

Knowledge of instructional strategies and practices that facilitate effective learning.

Instructional Strategies

Practical Applications

and Reflective Paper

Instructor grading The mean score is 18.9/20. The range is 0-20 out of 20 pts. A total of 47 students were in the course during these three

semesters.

MED 6080: Conducting Educational Research

Ability to analyze and critique educational research.

Literature Synthesis

and Critique


for MED 6080

The mean score is 87.15/100. The range is 0-100 out of 100 pts. A total of 41 students were in the course during fall &

spring semesters.

MED 6085: Proposal Writing

Ability to analyze and critique educational research. AND Ability to use writing to meet

scholarly and professional goals.

Literature Review

section of the Proposal


for MED 6085

Mastery Model

The mean score is 26/30. The range is 0-30 out of 30 pts. A total of 37 students were in the course during these three

semesters.

MED 6090: Master's Project

Ability to find and report on a significant educational question that has usefulness and

applicability through the development of a written

project. AND Ability to use writing to meet scholarly and professional goals.

Final Project/Thesis Rubric with final grade:

Mastery Model

Final Grades: 28 students graduated in the three terms. Of the 29, 22 received A’s on their final project; 4 received A-‘s; 2

received B+’s. Two other students were registered in 6090,

but as of Dec., 2014 have not completed their projects.

MED 6091: Graduate Synthesis Seminar

Ability to critically and reflectively synthesize

personal and professional experience in the graduate program through the development of the

project and the portfolio.

Synthesizing portfolio

of all core courses

with reflections

Scoring Scale: Portfolio* The mean score is 48/50. The range is 45-50. A total of 47

students were in the course during these three semesters.

*New rubrics (See below.)

5

Rubrics for Course Outcomes (rubrics that have not changed can be found in the 2011-2012 document):

MED 6050 – UbD (Understanding by Design) Unit Design Scoring Guide (Comments column deleted to save space.)

Stage One

On Target Partly Understood Needs Improvement

Introduction to Unit Introduction explains unit, the big ideas, and the concepts,

theories, and processes that serve as the focal point of the unit.

(5 points)

Introduction explains unit topic and the big

ideas.

(3 points)

Introduction restates the establish goals.

(1 point)

Standard(s) or

Established Goals

Standards are listed, including the subject, number and

narrative; if more than one standard is listed, they are all from

the same grade level. Standard drives unit.

(5 points)

X

Standards are not listed with all pieces (subject,

number, and narrative) or not listed.

(1 point)

Transfer Outcomes are created based upon long-term learning goals,

make reference to independent learning and align to the

standard. Thorough and completely represent standard.

(5 points)

Outcomes address deeper understanding but

are limited in content and/or limited in

thought about standard.

(3 points)

Deeper meaning or transfer of outcomes is not

evident and/or outcome is not aligned to the

standard.

(1 point)

Meaning

Essential Questions

Essential questions asked are ongoing and may not have a

black/white answer. Questions address controversial issues in

the field. have no “right” answer, are meant to be argued, provoke and sustain student inquiry, while focusing learning

and final performance, and written in age appropriate language.

Thorough and complete standard. (5 points)

Essential questions are basic and probably

answerable with a bit of learning or research.

(3 points)

Essential questions are not aligned to standard

or are simple, answered questions.

(1 point)

Meaning

Enduring Understandings

Understandings are written as statements with no redundancy.

Involve big ideas, are aligned with goals, both over arching and topical, and may provide a conceptual foundation for basic

skills.

(5 points)

Understandings are written as statements but

do not thoroughly represent standard.

(3 points)

Understandings are redundant with

knowledge/skill objectives.

(1 point)

Acquisition

Knowledge

The outcomes are measurable, student centered, concise and match standard. Outcomes are inclusive of most areas of the

standard. May include: vocabulary, facts, formulas, critical

details, important events and people. (5 points)

The outcomes are vague and don’t address the entire standard. They are limited in the

content. Outcomes are redundant with other

areas of the unit design. (3 points)

Outcomes are not student-centered or measurable. They don’t align with the

standard. The standard is not fully addressed

based upon the outcomes provided. (1 point)

Acquisition

Skills

The outcomes are expressed as a verb or verb phrase, address

six facets of understanding, basic skills of solving, communicating, thinking, researching and studying.

(5 points)

The outcomes are vague and don’t address the

entire standard. They are limited in the content or redundant with other areas of the

unit design. The outcomes don’t address the

six facets of understanding. (3 points)

Outcomes are not student-centered or

measurable. They don’t align with standard. They are redundant with knowledge or

understanding.

(1 point)

6

Stage Two


Summative Assessment

Blueprint for Assessments

A blueprint for assessment is included and completed

accurately with justification for unbalanced objectives (if

needed). Percentages relate to prioritization of importance. (5 points)

A blueprint for assessment is included but is

not accurate. The percentages don’t prioritize

the learning importance. (3 points)

Not all objectives are included or no blue print

is included.

(1 point)

Test Plan A test plan is included and completed accurately with

justification for percentage of objective weight. (5 points)

A test plan is included but is not accurate.

Objectives are unbalanced based upon percentages.

(3 points)

No test plan is included. Objectives are not

balanced based upon big idea of unit.

(1 point)


Valid and reliable

The summative assessments or task clearly provides a valid

measure of the targets understanding. Students must demonstrate the desired understanding to successfully complete

the task.

(5 points)

The summative assessments or task clearly

provides a valid measure of the targets understanding OR students must demonstrate

the desired understanding to successfully

complete the task, but not both. (3 points)

The summative assessments or tasks are not

reliable or valid.

(1point)


Design

Summative assessment(s) is/are designed using the guidelines

of the particular method you used. The choice of assessment type is appropriate for the standard/objectives tested.

(5 points)

Most of the assessment is designed correctly.

The choice of assessment type works, but there may be a different type that would be

better.

(3 points)

Summative assessment(s) is/are designed

without regard to the guidelines for that type of assessment and/or the choice of assessment

type is not appropriate for the

standard/objectives tested. (1 point)

Stage Three


WHERETO

W- Students will know where, why, what is required of them. H- Hook and hold student interest

E- Equip students with experiences, tools, knowledge and

strategies R- Students will rethink, reflect and revise

E- Students evaluate progress, and self-adjust

T-Tailor the unit to a diverse class O- Organize unit for engagement and effectiveness

(5 points)

On WHERETO 5out of the 7 components are not correct.

(3 points)

On WHERETO 3 out of the 7 components are not correct.

(1 point)

Acquisition, Meaning, and

Transfer Objectives aligned

with Teaching

Strategies/Meaningful Activities

Objectives align with teaching strategies that have students

learning in engaging and effective ways.

(5 points)

Objectives align but aren’t engaging and/or

effective.

(3 points)

Objectives aren’t aligned with teaching

strategies or meaningful activities.

(1 point)

Daily Lesson Plan

Objective/ meaningful

activity/ formative assessment

Daily lessons show a progression of learning that scaffolds

learning, and brings about learning outcomes. Objectives align

with unit outcomes, meaningful activities are engaging and effective, formative assessments allow learner and teacher to

make instructional decisions based upon learning.

(5 points)

One of the components does not promote

engagement, effectiveness or a progression of

learning or formative assessments doesn’t engage learner with self- assessment of

learning.

(3 points)

Daily lessons are not scaffold or activities do

not align with objectives or formative

assessment does not promote instructional decision-making.

(1 point)

7

MED 6091 - Portfolio Rubric 1. Student demonstrates with artifacts and discusses evidence of growth/change in writing ability during his/her program

Shows artifacts and explains/discusses growth/change in

writing.

7

Discusses growth/change in writing when asked.

5

Does not discuss or show artifacts.

0

2. Student demonstrates with artifacts and discusses evidence of growth/change in research abilities during his/her program.

Shows artifacts and explains/ discusses growth/change

in research skills.

7

Discusses growth/change in research skills when asked.

5


0

3. Student demonstrates and discusses reasons for the organization of the portfolio.

Shows artifacts and explains/discusses reasons for the

organization of the portfolio.

7

Discusses reasons for the organization of the portfolio

when asked.

5


0

4. Student demonstrates and discusses the table of content for the portfolio.

Shows artifacts and explains/discusses reasons for the

table of content.

7

Discusses reasons for the table of content when asked.

5


0

5. Student demonstrates and discusses the types and findings of the evaluations contained within the portfolio.

Shows artifacts and explains/

discusses the types and findings of the evaluations.

7

Discusses the types and findings of the evaluations when

asked. 5


0

6. All core classes have at least one artifact.

Yes.

5

No.

0

7. Overall Look of the Portfolio

The overall look and flow are effective. No grammar or

writing errors

10

The overall look and flow are adequate. Few grammar

or writing errors.

5

The overall look and flow are inadequate. Many errors.

0

Total points:______________/50

8

G. Summary of Artifact Collection Procedure

Each professor is responsible for keeping and assessing the data for his or her courses. The grading rubric for the final project

is kept in the individual student’s file in a secure filing cabinet. The files are kept for 5 years before they are shredded.

9

Appendix A

Report of progress on ‘non-learning-outcome recommendations’ from previous 5 year program review (optional):

Date of Program Review:

Feb. 11, 2014 Recommendation Progress Description

Recommendation 1

The M.Ed. and post-bacc licensure program need to be two

separate programs. The M.Ed. and post-bacc licensure

should have separate mission statements and sets of

learning outcomes.

The director is in the process of writing a

proposal to create a graduate certificate

for each of the three post-baccalaureate

licensing programs. It is hoped that the

new programs will be approved by the

Board of Regents during the summer of

2015.

The mission statement for the M.Ed. has

been altered; the mission statement for the

post-baccalaureate licensing has been

aligned to the undergraduate licensing

program as of August, 2014.

Recommendation 2

The program should create three sets of admission

criteria—a set for the undergraduate program, a set for the

post-bacc licensure program, and a set for the M.Ed.

program. Each program has a different mission and is

catering to a slightly different set of students. Having

criteria specifically for the M.Ed. that isn’t impacted by the

post-bacc. licensure will strengthen the program and allow

the faculty to select the best students to mentor in graduate

studies.

Undergraduate admissions will not be

addressed here.

Post-baccalaureate admissions have be

changed to reflect the State Board Rule

which requires a 3.0 cumulative GPA (we

base that on the students’ undergraduate

transcripts), and they must take and pass a

nationally normed test (we are using the

appropriate Praxis II tests for each

licensing area with the approval of the

USOE. Group interviews will be used for

those licensing similar to what is used in

the undergrad program. (August, 2014)

Criteria for admittance into the M.Ed.

program will remain the same.

10

Recommendation 3

The program should continue its efforts to transition to the

Utah Effective Teaching Standards (UETS). The program

should align curriculum and program assessments with the

standards.

Practicum and student teaching

observation/assessment forms will be

piloted during spring, 2015. Although it

was not specifically stated, this alignment

is focused on the licensing programs.

Recommendation 4

The program should evaluate ways that the post-bacc

licensure students can receive advisement that relieves

pressure from Dr. Saunders. The program indicated that

Lynda Goucher, part-time program secretary, does help

with this effort along with her other duties.

Post-bacc. licensure people do not receive

any more advisement from Dr. Saunders

that regular M.Ed. students. Unclear in

the document was the fact that they

receive advising from the M.Ed. admin.

asst., the student teaching coordinator, and

the advisor in charge of licensing.

Mrs. Goucher retired in Aug., 2014 and

was replaced by a full-time administrative

assistant who has 50% of her duties in

advising and the other 50% as the office

manager.

Recommendation 5

The program indicated that Lynda Goucher’s position,

part-time program secretary, should be reevaluated for an

increase in compensation commensurate with

responsibilities. The team further recommends that the

position be expanded to a full-time FTE providing

additional resource to support the M.Ed. program.

The Dean has authorized the position to be

moved to full-time in Aug., 2011;

however, Mrs. Goucher preferred her part-

time status. She was too good at her job

to remove her from the position. This

issue was rectified upon her retirement in

August, 2014. See progress note above.

Recommendation 6

The program is encouraged to reach out to their K-12 and

community partners and reconvene the Advisory

Committee that hasn’t met since spring 2011.

Although the Advisory Committee has not

met, the program director has met with

many community partners which were not

directly addressed in the program review.

These partnerships are more dynamic and

synergistic than the Advisory Committee.

The director meets with practicing

teachers, principals, human resource

directors throughout the school year.

11

Appendix B – Faculty

All faculty counts are part of the Teacher Education Department of which the Master of Education program is a part. As such, this

section does not apply.

Please respond to the following questions.

1) Reflecting on this year’s assessment(s), how does the evidence of student learning impact your faculty’s confidence in the

program being reviewed; how does that analysis change when compared with previous assessment evidence?

The outcomes for the program have always been exceptional. It is a great program with a long standing history. The post-

baccalaureate licensing programs have created an additional burden on the program and the faculty, yet the outcomes remain

strong. The learning by individuals has not been compromised at any time. It must be noted that none of the recommendations

from the 5-year program review, nor the national TEAC audit, had concerns about the learning outcomes of the program.

With that said, we are a faculty who is committed to continuous improvement. When something new is coming into the k-12

system, such as the Common Core State Standards, we modify our teaching objectives to reflect those changes. We are

committed to preparing teachers who have the knowledge, skills, and ethical behaviors that align with the Utah Effective

Teaching Standards.

Our “regular” master’s students are prepared to be better teachers and are tapped by schools and school districts to be teacher

leaders. They read, understand, and dissect the educational research. They conduct their own research and write a project/

thesis on their chosen topic. It is a rigorous program.

2) With whom did you share the results of the year’s assessment efforts?

The assessment results were shared with the MED Policy Committee, faculty members in the Teacher Education Dept., the

dean and associate dean of the Moyes College of Education, and various staff members. The results will also be available as a

link on the appropriate page of the Office of Institutional Effectiveness which anyone can access.

12

3) Based on your program’s assessment findings, what subsequent action will your program take?

The program will follow through on the creation of the graduate certificates for the post-baccalaureate licensing programs. We

will continually monitor new policies and rules that come from the Utah State Board of Education and modify our admission

process, educational outcomes, and assessments accordingly.

Weber State University Annual Assessment of Evidence …. Report of assessment results for the most...

Documents

Transcript of Weber State University Annual Assessment of Evidence …. Report of assessment results for the most...