Introduction NA.doc

MS Engineering/Science Management

Educational Effectiveness

Assessment Plan

2003-2004

Adopted by

The Engineering/Science Management faculty: Unknown Date

Submitted to

The Dean of the School of Engineering : Unknown Date The Office of Academic Affairs: Unknown Date

document.doc Submitted Fall 2003 of 22

TABLE OF CONTENTS

Introduction _________________________________________________________________NA

Program Objectives ___________________________________________________________NA

Program Outcomes ___________________________________________________________NA

Assessment Tools _____________________________________________________________NA

Program Objectives Results ____________________________________________________NA

Analysis of Objective Results ___________________________________________________NA

Program Outcomes Results _____________________________________________________NA

Analysis of Outcome Results ____________________________________________________NA

Appendix: Advisory Board Review _______________________________________________NA

Appendix: ESM Thesis or Final Project __________________________________________NA

Appendix: Alumni Survey ______________________________________________________NA

Appendix: Enrollment Growth __________________________________________________NA

Appendix: Student Course Evaluation and Comment Sheet ___________________________NA


INTRODUCTION

The mission of the Engineering/Science Management program is to provide state of the art management and leadership education for technical professionals, including engineers and scientists functioning within technology-oriented organizations, while creating new knowledge and disseminating new knowledge through research and professional activities.

The ESM currently offers two M.S. degrees:

• M.S. in Engineering Management• M.S. in Science Management

Both degrees follow the same curriculum with the distinction being the student’s undergraduate degree, i.e., engineering vs. non-engineering. In addition, a proposal is currently under review by the UAA Provost to establish a Masters of Science, Project Management degree. This degree is a natural extension of the Engineering Management program and is envisioned to be cross disciplinary (that is, reaching out to a multitude of undergraduate technological backgrounds) in nature.

This ESM program serves a unique group of students and its curriculum has been developed toward their needs. The students attend the program to learn technology-related management skills, while they continue to improve their technical (engineering and science major) capabilities. These students are engineering managers, research managers, project managers, first-time managers, etc. The ESM program’s courses and students are different from those of MBA in School of Business in that our focus is on management technology-oriented activities for the Engineering worker and manager, whereas the MBA focuses on non-technical management activities, such as accounting, investment or business development. Both are needed to form a balanced set of training courses at the university, and for the engineering community.

The ESM focus provides engineering-specific management science to the challenging, unique, and often hazardous discipline of engineering management. While the MBA does offer a great foundation for general management, to complete the engineering leadership training and education demands a balance of engineering economy and engineering knowledge management together with some of the accounting and human behavior aspects of the MBA. That is why there will always be a crossover of a few courses between the MBA and ESM programs. With such a cooperative approach to leadership training, the ESM student becomes a leader among their peers.

The ESM Advisory Board annually reviews the program and curriculum to ensure the communities needs are being met and to offer suggestion for improvement and change. In addition, student evaluations are given and analyzed each semester.

The ESM Program also went through an external review process which was finalized by the review committee March 2003. The provost and vice president of academic affairs along with the Dean and Chair of the department will meet in November 2003 to review the outcomes.


PROGRAM OBJECTIVES

It is the objective of the UAA Engineering/Science Management program to produce graduates that:

1. have a strong general engineering or science foundation and are able to use modern technological tools to solve complex multidisciplinary problems.

2. can effectively work in teams to solve real world problems through information research, with economic justification of the solution, using project management tools and techniques and effective written and/or communications.

3. are poised to assume leadership positions in their chosen discipline of work. 4. are able to apply engineering/science management resources to effectively plan, staff,

organize, and control projects for successful department operation. 5. will contribute lessons learned to advance knowledge in management areas of

engineering/science management.


PROGRAM OUTCOMES

The graduates of the UAA Engineering/Science Management program will:

1. Approach engineering and business problems from a real-world environmental and cultural perspective, rather than relying solely on a static textbook environment.

2. Focus on the fundamental management principles for significant, long-lasting improvements, instead of just dealing with day-to-day operational efficiency.

3. Seek systematic problem understanding and solutions, rather than simple reactive adjustments to individual components.

4. Be able to work in changing local, national, and international business environments.5. Be able to form an integrated problem statement from various, often uncertain,

ambiguous data and recommend decisions.6. Be a contributing member of the engineering management community by examining,

questioning, and providing alternatives to today’s management, business, and engineering challenges and issues.

7. Have team-building, leadership, and communication skills.8. Be able to integrate feasibility requirements with current conditions for using engineering

management resources.9. Be able to identify risks and offer solutions for minimizing risk. 10. Develop a life long commitment to learning and through the required design project form

the skills necessary research a new topic and apply new skills. In the fast paced changing business environment this learning skill may be the most valuable to the graduate.


ASSESSMENT TOOLS

A description of the tools used in the assessment of the program objectives and their implementation are summarized in Table 1. The tools and their relationships to the program objectives are listed in Table 2.

A description of the tools used in the assessment of the program outcomes and their implementation are summarized in Table 3. The tools and their relationships to the program outcomes are listed in Table 4.

There is a separate appendix for each tool that includes a more detailed description than is provided here and also describes the factors that affect the results and give examples of the tools and how they will be implemented.

Table 1Program Objectives Assessment Tools and Administration

Tool DescriptionFrequency/ Start Date

Collection Method

Administered by

Advisory Board

ReviewAnnual Advisory Board Meeting

Administered yearly

beginning February 2002

In person, phone, or

email

Program Faculty and

Staff

ESM Thesis or

Final Project

Evaluation of student performance relative to program outcomes in the

execution of their thesis or final project

Annually beginning

Spring 2004

Program Faculty

Committee

Instructor and Peers

Alumni Survey

Alumni will be surveyed one and three years after graduation to evaluate program and course

objectives

One and three years after graduation beginning

Spring 2004

Online surveySchool of

Engineering

Enrollment Growth

Semester Enrollment StatisticsSemester, beginning

Spring 2004

Student Enrollment

Staff


Table 2Association of Assessment Tools to Program Objectives

0 = Tool is not used to measure the associated objective.1 = Tool is used to measure the associated objective.

Advisory Board

Review

ESM Thesis or

Final Project

Alumni Survey

Enrollment Growth

have a strong general engineering or science foundation and are able to use modern technological tools to solve complex multidisciplinary problems.

1 1 0 1

can effectively work in teams to solve real world problems through information research, with economic justification of the solution, using project management tools and techniques and effective written and/or communications.

1 1 0 1

are poised to assume leadership positions in their chosen discipline of work. 1 1 1 1are able to apply engineering/science management resources to effectively plan, staff, organize, and control projects for successful department operation.

1 1 1 1

will contribute lessons learned to advance knowledge in management areas of engineering/science management.

1 0 0 1

document.doc Submitted Fall 2003 Page 9 of 22

Table 3Program Outcomes Assessment Tools and Administration

Tool DescriptionFrequency/ Start Date

Collection Method

Administered by

ESM Thesis or

Final Project

Evaluation of student performance relative to program outcomes in the

execution of their thesis or final project

Annually beginning

Spring 2004

Program Faculty

Committee

Instructor and Peers

Alumni Survey

Alumni will be surveyed one and three years after graduation to evaluate program and course

objectives

One and three years after graduation beginning

Spring 2004

Online surveySchool of

Engineering

Student Course

Evaluation and

Comment Sheet

Evaluation at semester end of instructor and course objective

achievement

Each semester at semester end starting

Fall 2003

Student Evaluation by

student

School of Engineering


Table 4Association of Assessment Tools to Program Outcomes

0 = Tool is not used to measure the associated objective.1 = Tool is used to measure the associated objective.

ESM Thesis or

Final Project

Alumni Survey

Student Course

Evaluation and

Comment Sheet

Approach engineering and business problems from a real-world environmental and cultural perspective, rather than relying solely on a static textbook environment.

1 0 1

Focus on the fundamental management principles for significant, long-lasting improvements, instead of just dealing with day-to-day operational efficiency.

1 1 1

Seek systematic problem understanding and solutions, rather than simple reactive adjustments to individual components.

1 0 1

Be able to work in changing local, national, and international business environments. 1 1 1Be able to form an integrated problem statement from various, often uncertain, ambiguous data and recommend decisions.

1 0 1

Be a contributing member of the engineering management community by examining, questioning, and providing alternatives to today’s management, business, and engineering challenges and issues.

1 1 1

Have team-building, leadership, and communication skills. 1 1 1Be able to integrate feasibility requirements with current conditions for using engineering management resources.

1 0 1

Be able to identify risks and offer solutions for minimizing risk. 1 0 1Develop a life long commitment to learning and through the required design project form the skills necessary research a new topic and apply new skills. In the fast paced changing business environment this learning skill may be the most valuable to the graduate.

1 1 1


Program Objectives AssessmentProgram: BS Civil EngineeringDepartment: Engineering/Science ManagementCollege/School: School of EngineeringAcademic Year: 2003-2004CIP Coordinator: 18

Program Objective Assessment Results

Advisory Board

Review

ESM Thesis

or Final Project

Alumni Survey

Enrollment Growth

Weighted Average

1

have a strong general engineering or science foundation and are able to use modern technological tools to solve complex

multidisciplinary problems.

Grade NG NG

NG

NGValue NG NG NG NG

Weight 1.0 1.0 1.0

2

can effectively work in teams to solve real world problems through information research,

with economic justification of the solution, using project management tools and

techniques and effective written and/or communications.

Grade NG NG

NG

NGValue NG NG NG NG

Weight 1.0 1.0 1.0

3are poised to assume leadership positions in

their chosen discipline of work.

Grade NG NG NG NGValue NG NG NG NG NG

Weight 1.0 1.0 1.0 1.0

4

are able to apply engineering/science management resources to effectively plan,

staff, organize, and control projects for successful department operation.

Grade NG NG NG NGValue NG NG NG NG NG

Weight 1.0 1.0 1.0 1.0

5will contribute lessons learned to advance

knowledge in management areas of engineering/science management.

Grade NGNG NG

NGValue NG NG NG

Weight 1.0 1.0


Program Objectives Assessment DiscussionProgram: BS Civil EngineeringDepartment: Engineering/Science ManagementCollege/School: School of EngineeringAcademic Year: 2003-2004CIP Coordinator: 18

Analysis of Objective Assessment Results

ObjectiveResul

tExplanation of

Results

1have a strong general engineering or science foundation and are able to use modern technological tools

to solve complex multidisciplinary problems. NG

No explanation given.

2can effectively work in teams to solve real world problems through information research, with

economic justification of the solution, using project management tools and techniques and effective written and/or communications.

NGNo explanation

given.

3 are poised to assume leadership positions in their chosen discipline of work. NGNo explanation

given.

4are able to apply engineering/science management resources to effectively plan, staff, organize, and

control projects for successful department operation.NG


5will contribute lessons learned to advance knowledge in management areas of engineering/science

management. NG



Program Outcomes AssessmentProgram: BS Civil EngineeringDepartment: Engineering/Science ManagementCollege/School: School of EngineeringAcademic Year: 2003-2004CIP Coordinator: 18

Program Outcomes Assessment Results

ESM Thesis or

Final Project

Alumni Survey

Student Course Evaluation and

Comment Sheet

Weighted Average

1

Approach engineering and business problems from a real-world environmental and cultural perspective,

rather than relying solely on a static textbook environment.

Grade NG

NG

NG FValue NG NG

Weight NG NG

2Focus on the fundamental management principles for significant, long-lasting improvements, instead of just

dealing with day-to-day operational efficiency.

Grade NG NG NG FValue NG NG NG

Weight NG NG NG

3Seek systematic problem understanding and solutions, rather than simple reactive adjustments to individual

components.

Grade NGNG

NG FValue NG NG

Weight NG NG

4Be able to work in changing local, national, and

international business environments.


Weight NG NG NG

5Be able to form an integrated problem statement from

various, often uncertain, ambiguous data and recommend decisions.

Grade NGNG

NG FValue NG NG

Weight NG NG

6

Be a contributing member of the engineering management community by examining, questioning, and providing alternatives to today’s management, business, and engineering challenges and issues.


Weight NG NG NG


7Have team-building, leadership, and communication

skills.


Weight NG NG NG

8Be able to integrate feasibility requirements with

current conditions for using engineering management resources.

Grade NGNG

NG FValue NG NG

Weight NG NG

9Be able to identify risks and offer solutions for

minimizing risk.

Grade NGNG

NG FValue NG NG

Weight NG NG

10

Develop a life long commitment to learning and through the required design project form the skills

necessary research a new topic and apply new skills. In the fast paced changing business environment this

learning skill may be the most valuable to the graduate.


Weight NG NG NG


Program Outcomes Assessment DiscussionProgram: BS Civil EngineeringDepartment: Engineering/Science ManagementCollege/School: School of EngineeringAcademic Year: 2003-2004CIP Coordinator: 18

Analysis of Outcome Assessment Results

Outcome ResultExplanation of

Results

1Approach engineering and business problems from a real-world environmental and cultural

perspective, rather than relying solely on a static textbook environment.F


2Focus on the fundamental management principles for significant, long-lasting improvements, instead

of just dealing with day-to-day operational efficiency.F


3Seek systematic problem understanding and solutions, rather than simple reactive adjustments to

individual components.F


4 Be able to work in changing local, national, and international business environments. FNo explanation

given.

5Be able to form an integrated problem statement from various, often uncertain, ambiguous data and

recommend decisions.F


6Be a contributing member of the engineering management community by examining, questioning, and

providing alternatives to today’s management, business, and engineering challenges and issues. F


7 Have team-building, leadership, and communication skills. FNo explanation

given.

8Be able to integrate feasibility requirements with current conditions for using engineering management

resources.F


9 Be able to identify risks and offer solutions for minimizing risk. FNo explanation

given.


10

Develop a life long commitment to learning and through the required design project form the skills necessary research a new topic and apply new skills. In the fast paced changing business environment

this learning skill may be the most valuable to the graduate.F



APPENDIX: ADVISORY BOARD REVIEW

Tool Description:

The Engineering/Science Management (ESM) Advisory Board consists of industry leaders in many area’s of ESM. Their knowledge and expertise will keep the department focused on the changing needs of the project management community.

The Advisory Board members were instrumental in the development of the program offering many suggestions for program content. As practicing professionals, they are able to give extreme insight into the areas of project management that need more support in which to ensure program objectives are met.

The advisory board meets annually at a half day seminar. Meetings with select Board members occur as needed throughout the year, depending on the particular topic to be discussed in the program. Additionally, phone or email surveys occur on an as needed basis if a particular issue regarding the program is to be decided.

Factors that affect the collected data:

The Advisory Board members comprise many different areas of business and industry where engineering and science management is practiced. There may be other areas of engineering and science management that may not be represented in our board members.

How to interpret the data:

Input from Advisory Board members should be discussed and outcomes should be compared against ESM needs by local project management associations, State ESM needs, and National and Global trends in ESM.

Sample Survey.

A sample survey is provided on the next page.

Tabulating and Reporting Results

Results will be gathered via email or in person at annual advisory board meetings and evaluated by faculty. Overall review will take place by the advisory board.


APPENDIX: ESM THESIS OR FINAL PROJECT

Tool Description:

The instructor survey evaluates the student’s performance against program outcomes. The thesis or final project integrates the knowledge, concepts, and skills associated with the entire sequence of study in the program. A senior or graduate student thesis or research project provides the student an opportunity to demonstrate a mastery of an array of skills and knowledge appropriate to ESM as well as to their specific study area of management.



Instructor and peer evaluations will be discussed and interpreted to determine if specific program knowledge and skills-based areas have been achieved.

Project or thesis evaluation will be examined. Suggestions will be made for program improvement as related to student's research and thesis outcomes.

Sample Survey.



Evaluation scores will be gathered by staff and tabulated and results given to faculty for review and discussion.


APPENDIX: ALUMNI SURVEY

Tool Description:

An online survey will ask graduates to evaluate the program and course objectives as related to their individual work environments. These graduates are experienced managers, many with 10 plus years of on the job experience. Their evaluation will be greatly weighted.


Some graduates may not respond to the survey. The data may not reflect all areas of project management they do not respond.


The survey results should be compiled and discussed among faculty members. These results should be compared with student surveys to see if they are inline with alumni evaluations.


The survey is prepared by faculty, administered and collected online. Staff will tabulate the result and submit them for review to faculty.


APPENDIX: ENROLLMENT GROWTH

Tool Description:

Student enrollment in courses with particular interest in student credit hours (SCH) will show if the program is inline with student project management needs. Increase in growth will show program objects are being met. Decrease in enrollments will show a disinterest in the program and program and course objectives will need to be reviewed.


Student enrollment may be capped due to student maximum capacity per classroom. Available classroom size may dictate enrollment growth.Students may wish to enroll in a course but because of financial situations are not able. This would show a decrease in enrollment, not related to program interest or quality.


Data should be compared on a semester basis with previous semester Program enrollments. Outcomes should be compared with student semester course evaluations. Outcomes should also be compared with overall UA and UAA enrollments.

Sample Survey.



Results will be obtained from UA enrollment records and tabulated by staff. Faculty will discuss and outcomes used in the program review.


APPENDIX: STUDENT COURSE EVALUATION AND COMMENT SHEET

Tool Description:

Students will determine whether or not course objectives have been met. Our students are practicing professionals, many having management jobs for more than 10 years. These professionals will offer valuable insight to course weaknesses and be able to offer suggestions for improvements.


Some students may not be present when the evaluation is given. In addition, the survey is voluntary and some students may choose not to participate.


The survey should be reviewed for course weaknesses and suggestions for change and improvements be discussed with faculty and advisory board members. These results should be compared with current with educational and community needs as perceived by the faculty and the advisory board.


Results will be tabulated via school staff. The attachment to the survey will be sent to the department chair and the instructor for evaluation.


Introduction NA.doc

Education

Transcript of Introduction NA.doc