Annual Academic Assessment Report Cover Sheet · The MBA teaching faculty and MBA Programs office...

19
10/2014 1 Annual Academic Assessment Report Cover Sheet Assessment reports are due the 1 st Wednesday after the Fall Term Email to: [email protected] Program Information: Program Assessed Master of Business Administration (MBA) Department MBA Programs Office College Lee Business School (LBS) Department Chair Chin-Chun (Vincent) Hsu, Ph.D., CMA Assessment Coordinator Chin-Chun (Vincent) Hsu, Ph.D., CMA Date Submitted December 31, 2018 Contact Person for This Report Name Chin-Chun (Vincent) Hsu, Ph.D., CMA Phone 895-3842 Email [email protected] Please attach a narrative (not to exceed 4 pages, excluding appendices) addressing the following: What are the student learning outcomes? Please provide a numbered list. Which learning outcomes were assessed? How were they assessed? (Programs must use at least one direct assessment of student learning.) Undergraduate programs should assess at least one University Undergraduate Learning Outcome (UULO) each year, which may or may not overlap with a program learning outcome. Graduate programs should assess at least one outcome related to one of the following graduate level requirements each year: o student engagement in research, scholarship, creative expression and/or appropriate high-level professional practice. o activities requiring originality, critical analysis and expertise. o the development of extensive knowledge in the field under study. What was learned from the assessment results? How did the program respond to what was learned? Please limit the narrative portion of your report to no more than four pages. You may attach appendices with data, tables, charts, or other materials as needed. Please explain the relevant conclusions from any appendices in your narrative. Please contact the Office of Academic Assessment if you have questions or need assistance.

Transcript of Annual Academic Assessment Report Cover Sheet · The MBA teaching faculty and MBA Programs office...

10/2014 1

Annual Academic Assessment Report Cover Sheet Assessment reports are due the 1st Wednesday after the Fall Term

Email to: [email protected] Program Information:

Program Assessed

Master of Business Administration (MBA)

Department

MBA Programs Office

College

Lee Business School (LBS)

Department Chair Chin-Chun (Vincent) Hsu, Ph.D., CMA

Assessment Coordinator

Chin-Chun (Vincent) Hsu, Ph.D., CMA

Date Submitted December 31, 2018

Contact Person for This Report

Name Chin-Chun (Vincent) Hsu, Ph.D., CMA

Phone 895-3842

Email

[email protected]

Please attach a narrative (not to exceed 4 pages, excluding appendices) addressing the following:

What are the student learning outcomes? Please provide a numbered list.

Which learning outcomes were assessed?

How were they assessed? (Programs must use at least one direct assessment of student learning.)

Undergraduate programs should assess at least one University Undergraduate Learning Outcome (UULO) each year, which may or may not overlap with a program learning outcome.

Graduate programs should assess at least one outcome related to one of the following graduate level requirements each year:

o student engagement in research, scholarship, creative expression and/or appropriate high-level professional practice.

o activities requiring originality, critical analysis and expertise. o the development of extensive knowledge in the field under study.

What was learned from the assessment results?

How did the program respond to what was learned?

Please limit the narrative portion of your report to no more than four pages. You may attach appendices with data, tables, charts, or other materials as needed. Please explain the relevant conclusions from any appendices in your narrative. Please contact the Office of Academic Assessment if you have questions or need assistance.

10/2014 2

What are the student learning outcomes? Please provide a numbered list.

A key aspect of the MBA Programs mission centers on helping students gain the necessary market-driven knowledge and professional skills to succeed in contemporary and future business environments. Toward this end, five learning goals spanning a broad range of knowledge and skill areas motivate the design and structure of the MBA programs. Specifically, upon completion of the MBA program, students should be able to:

1. SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business

decisions

2. SLO 2: Communicate effectively business ideas and analyses in writing

3. SLO 3: Communicate effectively business ideas and analyses in oral presentations

4. SLO 4: Work effectively in a team setting

5. SLO 5: Apply business concepts to managerial decision making

Which learning outcomes were assessed?

Annually, the MBA Programs Office completes the following assessment process for each student learning outcome.

1. Collect course artifacts from appropriate MBA curriculum courses

2. Assess performance of course artifacts against learning goal performance standards

3. Identify learning goal areas of sub-par performance

4. Recommend actions to raise performance in sub-par areas to par or higher levels.

5. Present results and recommendations to MBA faculty. In 2018, we did the assessment by following the curriculum map in the newly developed 2015-2018 Three-year Assessment Plan. In 2018, we assessed all SLOs through several groups of students:

We assessed SLO 1 through MBA 787

We assessed SLO 2 through MBA 763 and MBA 767.

We assessed SLO 3 through MBA 787.

We assessed SLO 4 through MBA 787.

We assessed SLO 5 through MBA 787.

How were they assessed? (Programs must use at least one direct assessment of student learning.)

The methodology relies heavily on course-embedded assessment instruments to generate course artifacts reflective of program progress toward the learning goals. Utilizing the course-

10/2014 3

embedded system of assessment will allow for detailed outcome-level data on achievement of the outcomes connected to each program goal. The MBA teaching faculty and MBA Programs office developed rubrics and assessment data collection forms to assess student achievement of the learning outcomes. Please refer to Appendix 1-5 for sample rubrics and sample forms. The forms identify the performance levels in the following fashion – Level 5 (Excellent), Level 4 (Good), Level 3 (Adequate), Level 2 (Weak), and Level 1 (Poor). We assessed SLO 1 through core course MBA 787 [Direct Assessment: course-embedded assessment].

MBA students will recognize and evaluate ethical, legal and regulatory dimensions of

business decisions.

Written ethics case analyses in MBA 787 (Capstone: Strategic Management) rated

against 5 criteria on 5-point scale: (1) Identification of Ethical Issues, (2) Consideration of

Stakeholders, (3) Analysis of Issue from Multiple Ethical Perspectives, (4) Discussion of

Options, and (5) Proposal of Decision. [See Appendix 1].

We assessed SLO 2 through core course MBA 763 and MBA 767 [Direct Assessment: course-embedded assessment].

MBA students will demonstrate ability of communicating effectively business ideas and

analyses in writing.

Written case analyses in MBA 763 (Leadership and Team) and MBA 767 (Marketing

Opportunity Analysis) rated against 4 writing criteria on 5-point scale: (1) Clarity &

Conciseness, (2) Professional Tone, (3) Organization, and (4) Grammar & Mechanics.

[See Appendix 2].

We assessed SLO 3 through core course MBA 787 [Direct Assessment: course-embedded assessment].

MBA students will demonstrate ability of communicating effectively business ideas and

analyses in oral presentation.

20-30 minute oral presentations of group project results in MBA 787 (Capstone:

Strategic Management) rated against 6 verbal communication criteria on 5-point scale:

(1) Organization, (2) Subject Knowledge, (3) Graphics, (4) Mechanics, (5) Eye Contact,

and (6) Elocution. [See Appendix 2].

10/2014 4

We assessed SLO 4 through core course MBA 787 [Indirect Assessment].

SLO 4 is to measure the student’s ability to work effectively in a team setting.

Students in MBA 787 (Capstone: Strategic Management) complete a 10-item

“Group Processes and Peer Evaluation Forms” questionnaire reflective of group

and individual-level facets of team work. (See Appendix 4).

This form was intended to measure the degree of effectiveness of the groups and the

extent to which all members did their share of the work.

We assessed SLO 5 through core course MBA 787 [Direct Assessment: course-embedded assessment].

SLO 5 is to measure the student’s ability to apply business concepts to managerial

decision making.

As the culminating experience for all Lee Business School’s MBA students, MBA 787

Capstone Business Projects give students the opportunity to demonstrate mastery of

the skills and knowledge gained throughout their program.

Project teams are made up of 4-5 students who are nearing the end of their program.

Teams contribute 600 to 900 hours of research and consulting work on the project,

working closely with the client company and their faculty advisor. Teams provide the

client with a presentation of their findings and recommendations. Presentations of

capstone business projects for client companies in MBA 787 rated by 5-10 Las Vegas

business executives and several graduate faculty members. (See Appendix 5).

Undergraduate programs should assess at least one University Undergraduate Learning Outcome (UULO) each year, which may or may not overlap with a program learning outcome.

Not Applicable. The Master of Business Administration (MBA) degree is a business graduate program. UULO does not apply to graduate programs.

Graduate programs should assess at least one outcome related to one of the following graduate level requirements each year: o student engagement in research, scholarship, creative expression and/or appropriate high-level

professional practice. o activities requiring originality, critical analysis and expertise. o the development of extensive knowledge in the field under study

The Master of Business Administration (MBA) degree is a non-thesis program designed to serve students, alumni, and businesses by providing educational programs that enhance leadership and competence among business people, especially business practitioners, and provides solutions for important business problems.

10/2014 5

What was learned from the assessment results?

We assessed SLO 1 through capstone course MBA 787 Results:

1. Table 1 and Table 2 report the results from MBA 787 Strategic Management. The SLO 1

were assessed using assessment rubric with five-point scale. As shown in Table 1, the

median scores for all five dimensions are in 4 or better.

2. Table 2 shows the percentage of students meeting and exceeding expectation. In terms

of Identification of Ethical Issues, most students do very well; 100% of students have

reached “Good” or “Excellent” levels. In terms of Consideration of Stakeholders, most

students do well with 100% of students have reached “Good” or “Excellent” levels. In

terms of Analysis of Issue from Multiple Ethical Perspectives, 100% of students have

reached “Good” or “Excellent” levels. In terms of Discussion of Options, 100% of

students have reached “Good” or “Excellent” levels. In terms of Proposal of Decision,

100% of students have reached “Good” or “Excellent” levels.

Table 1 Summary Statistics

SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business decisions

Rubric Mean Median St.Dev. Min Max N

Identification of Ethical Issues 4.17 4 0.41 4 5 16

Consideration of Stakeholders 4.83 4 0.41 4 5 16

Analysis of Issue from Multiple Ethical Perspectives 4.33 4 0.52 4 5 16

Discussion of Options 4.50 4 0.55 4 5 16

Proposal of Decision 4.33 4 0.52 4 5 16

10/2014 6

Table 2 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business decisions

Rubric Excellent

[5]

Good

[4]

Adequate

[3]

Weak

[2]

Poor

[1]

Identification of Ethical Issues 17 % 83 % 0 % 0 % 0 %

Consideration of Stakeholders 83 % 17 % 0 % 0 % 0 %

Analysis of Issue from Multiple Ethical Perspectives 34 % 66 % 0 % 0 % 0 %

Discussion of Options 50 % 50 % 0 % 0 % 0 %

Proposal of Decision 34 % 66 % 0 % 0 % 0 %

We assessed SLO 2 through core course MBA 763 and MBA 767. Results:

1. Table 3 and Table 4 report the results from MBA 763 Leadership, Teams and Individuals.

The SLO 2 were assessed using assessment rubric with five-point scale.

2. Table 4 shows the percentage of students meeting and exceeding expectation. In terms

of Clarity and Conciseness, most students do very well; 85% of students have reached

“Good” or “Excellent” levels. In terms of Professional Tone, 85% of students have

reached “Good” or “Excellent” levels. In terms of Organization, 80% of students have

reached “Good” or “Excellent” levels. In terms of Grammar and Mechanics 85% of

students have reached “Good” or “Excellent” levels.

Table 3 Summary Statistics

SLO 2: Communicate effectively business ideas and analyses in writing

Rubric Mean Median St.Dev. Min Max N

Clarity And Conciseness 4.21 4 .70 3 5 39

Professional Tone 4.21 4 .70 3 5 39

Organization 4.08 4 .77 2 5 39

Grammar and Mechanics 4.15 4 .81 2 5 39

10/2014 7

Table 4 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 2: Communicate effectively business ideas and analyses in writing

Rubric Excellent

[5]

Good

[4]

Adequate

[3]

Weak

[2]

Poor

[1]

Clarity And Conciseness 36 % 49 % 15 % 0 % 0 %

Professional Tone 36 % 49 % 15 % 0 % 0 %

Organization 31 % 49 % 17 % 3 % 0 %

Grammar and Mechanics 36 % 49 % 10 % 5 % 0 %

Results:

1. Table 5 and Table 6 report the results from MBA 767 Market Opportunity Analysis. The

SLO 2 were assessed using assessment rubric with five-point scale. As shown in Table 5,

the median scores for all five dimensions are in 4.

2. Table 6 shows the percentage of students meeting and exceeding expectation. In terms

of Clarity and Conciseness, most students do very well with a median of 4.0; 77% of

students have reached “Good” or “Excellent” levels. In terms of Professional Tone, most

students do well with a median score of 4 with 93% of students have reached “Good” or

“Excellent” levels. In terms of Organization, the median score is 4.0 and 53% of students

have reached “Good” or “Excellent” levels. In terms of Grammar and Mechanics, the

median score is 4.0 and 69% of students have reached “Good” or “Excellent” levels.

Table 5 Summary Statistics

SLO 2: Communicate effectively business ideas and analyses in writing

Rubric Mean Median St.Dev. Min Max N

Clarity And Conciseness 3.85 4 .15 3 5 13

Professional Tone 4.25 4 .18 3 5 13

Organization 3.69 4 .21 3 5 13

Grammar and Mechanics 3.69 4 .13 3 4 13

10/2014 8

Table 6 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 2: Communicate effectively business ideas and analyses in writing

Rubric Excellent

[5]

Good

[4]

Adequate

[3]

Weak

[2]

Poor

[1]

Clarity And Conciseness 8 % 69 % 23 % 0 % 0 %

Professional Tone 31 % 62 % 7 % 0 % 0 %

Organization 15 % 38 % 47 % 0 % 0 %

Grammar and Mechanics 0 % 69 % 31 % 0 % 0 %

We assessed SLO 3 through core course MBA 787. Results:

1. Table 7 and Table 8 report the results from MBA 787 Accounting for Managers. The

SLO 3 were assessed using assessment rubric with five-point scale. As shown in Table 7,

the median scores for all six dimensions are in 4 or better.

2. Table 8 shows the percentage of students meeting and exceeding expectation. In terms

of Organization, most students do very well with a median of 5.0; 95% of students have

reached “Good” or “Excellent” levels. In terms of Subject Knowledge, most students do

well with a median score of 5.0 with 100% of students have reached “Good” or

“Excellent” levels. In terms of Graphics, the median score is 5.0 and 92% of students

have reached “Good” or “Excellent” levels. In terms of Mechanics, the median score is

5.0 and 100% of students have reached “Good” or “Excellent” levels. In terms of Eye

Contact, the median score is 4.0 and 77% of students have reached “Good” or

“Excellent” levels. In terms of Elocution, the median score is 5.0 and 96% of students

have reached “Good” or “Excellent” levels.

10/2014 9

Table 7 Summary Statistics

SLO 3: Communicate effectively business ideas and analyses in oral communication

Rubric Mean Median St.Dev. Min Max N

Organization 4.63 5 0.50 4 5 16

Subject Knowledge 4.44 5 0.51 4 5 16

Graphics 4.38 5 0.62 4 5 16

Mechanics 4.44 5 0.51 4 5 16

Eye Contact 4.56 5 0.51 4 5 16

Elocution 4.38 5 0.50 4 5 16

Table 8 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 3: Communicate effectively business ideas and analyses in oral communication

Rubric Excellent

[5]

Good

[4]

Adequate

[3]

Weak

[2]

Poor

[1]

Organization 63 % 37 % 0 % 0 % 0 %

Subject Knowledge 44 % 56 % 0 % 0 % 0 %

Graphics 44 % 49 % 7 % 0 % 0 %

Mechanics 44 % 56 % 0 % 0 % 0 %

Eye Contact 56 % 44 % 0 % 0 % 0 %

Elocution 38 % 62 % 0 % 0 % 0 %

10/2014 10

We assessed SLO 4 through core course MBA 787. Results:

1. Table 9 and Table 10 report the results from MBA 787 Capstone Project assessment.

The SLO 4 were assessed using assessment rubric with five-point scale (5 = strongly

agree and 1 = strongly disagree).

2. The students were asked to rate their team’s performance. The percentage of students

rating their team as “strongly agree” or “agree” for each question is shown in Table 10.

It should be noted that the results represent all students who completed ratings for

each question. As shown in Table 9, the median scores for all 10 questions are in 5.

Table 9 Summary Statistics

SLO 4: Wok effectively in a team setting

Mean Median St.Dev. Min Max N

Our group worked as a team. 4.80 5 0.77 2 5 15

We discuss group norms. 4.40 5 0.91 2 5 15

We had an initial group meeting get organized. 4.53 5 1.06 1 5 15

We assign individual responsibilities. 4.80 5 0.77 2 5 15

Every group member did their share of the work. 4.87 5 0.35 4 5 15

Group members fulfilled their responsibilities as assigned. 4.87 5 0.52 3 5 15

Group members were prepared for group meetings. 4.73 5 0.80 2 5 15

Contributions of group members met my expectations. 4.73 5 1.03 1 5 15

The final presentation met my expectations. 4.73 5 1.03 1 5 15

I could not do a better job if this were an individual assignment. 4.53 5 1.13 1 5 15

10/2014 11

Table 10 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 4: Work effectively in a team setting

Strong Agree

[5]

Agree

[4]

Cannot Decide

[3]

Disagree

[2]

Strongly Disagree

[1]

Our group worked as a team. 93 % 0 % 0 % 7 % 0 %

We discuss group norms. 60 % 26 % 7 % 7 % 0 %

We had an initial group meeting get organized. 73 % 20 % 0 % 0 % 7 %

We assign individual responsibilities. 93 % 0 % 0 % 7 % 0 %

Every group member did their share of the work. 87 % 13 % 0 % 0 % 0 %

Group members fulfilled their responsibilities as assigned. 93 % 0 % 7 % 0 % 0 %

Group members were prepared for group meetings. 87 % 7 % 0 % 7 % 0 %

Contributions of group members met my expectations. 93 % 0 % 0 % 0 % 7 %

The final presentation met my expectations. 93 % 0 % 0 % 0 % 7 %

I could not do a better job if this were an individual assignment. 80 % 7 % 7 % 0 % 7 %

10/2014 12

We assessed SLO 5 through core course MBA 787. Results:

1. Table 11 and Table 12 report the results from MBA 787 Capstone Project assessment.

The SLO 5 were assessed using assessment rubric with five-point scale (5 = strongly

agree and 1 = strongly disagree).

Table 11 Summary Statistics

SLO 5: Apply business concepts to managerial decision making

Rubric Mean Median St.Dev. Min Max N

Identifies & Summarizes Problem at Issue 4.50 5 0.55 4 5 16

Considers Stakeholders 4.50 5 0.55 4 5 16

Evaluation 4.50 5 0.55 4 5 16

Application of Analysis to Management Situation 4.33 5 0.52 4 5 16

Chooses an Action 4.33 5 0.52 4 5 16

Table 12 Assurance of Learning

(Percentage of Students Achieving Learning Goals)

SLO 5: Apply business concepts to managerial decision making

Rubric Excellent

[5]

Good

[4]

Adequate

[3]

Weak

[2]

Poor

[1]

Identifies & Summarizes Problem at Issue 40 % 60 % 0 % 0 % 0 %

Considers Stakeholders 60 % 40 % 0 % 0 % 0 %

Evaluation 20 % 80 % 0 % 0 % 0 %

Application of Analysis to Management Situation 20 % 80 % 0 % 0 % 0 %

Chooses an Action 60 % 40 % 0 % 0 % 0 %

10/2014 13

How did the program respond to what was learned?

1. The MBA assessment (assurance of learning) process is valuable for learning what areas of

improvement are needed to ensure MBA students are learning what we believe they are

learning. The assessment is performed using course-embedded techniques.

2. Utilizing the course-embedded system of assessment will allow for detailed outcome-level

data on achievement of the outcomes connected to each program goal. The MBA

Assessment Committee develops assessment forms and rubrics, and the faculty members

teaching each of the courses identified in the Curriculum map incorporate assessment tasks

into their courses. All outcomes are assessed in required courses identified in the

Curriculum Map.

3. The program-level benchmark for Excellence is set for 100 percent of students that

completed the MBA program with a passing assessment to achieve Level 5 [Excellence] or

Level 4 [Good] performance in all SLOs. The program-level benchmark for satisfactory

performance is set for 80 percent of students that completed the course with a passing

assessment to achieve Level 5 [Excellence] or Level 4 [Good] performance in all SLOs.

4. Overall speaking, we are meeting goal however greater emphasis on business decision-

making (SLO 5) is desired in the program. The faculty are currently considering ways to

improve the assessment of this learning goal in the overall program.

5. Summaries of assessment results from 2015 to 2018 are presented here. Overall assessment

Scheme: “Fails to Meet Criterion” if Mean Rating = 1 or 2, “Marginally Meets Criterion” if

Mean Rating >2 and <4, “Meets Criterion” if Mean Rating = 4 or 5.

10/2014 14

MBA Program Assessment Results (2015-2018) SLO #1: Recognize & Evaluate Ethical, Legal, & Regulatory Dimensions of Business Decisions

Criterion 2015 2016 2017 2018 Assessment

Results

Identification of Ethical Issues 2.7 3.8 3.4 4.17 Meets

Consideration of Stakeholders 2.7 4.1 3.4 4.83 Meets

Analysis of Issue from Multiple Ethical Perspectives

2.7 4.0 4.0 4.33 Meets

Discussion of Options 2.7 4.0 3.3 4.50 Meets

Proposal of Decision 2.7 4.3 3.0 4.33 Meets

MBA Program Assessment Results (2015-2018) SLO #2 Communicate Effectively Business Ideas & Analyses in Writing

Criterion 2015 2016 2017 2018 Assessment

Results Clarity & Conciseness 4.4 3.7 4.2 4.21 Meets Professional Tone 4.4 4.1 4.1 4.21 Meets Organization 4.4 3.9 4.6 4.08 Meets Grammar & Mechanics 4.4 3.8 4.3 4.15 Meets

MBA Program Assessment Results (2015-2018) SLO #3: Communicate Effectively Business Ideas & Analyses in Oral Presentation

Criterion 2015 2016 2017 2018 Assessment

Results Organization 4.4 4.7 4.6 4.63 Meets Subject Knowledge 4.4 4.7 4.4 4.44 Meets Graphics 4.4 4.6 NA 4.38 Meets Mechanics 4.4 5.0 4.9 4.44 Meets Eye Contact 4.4 4.0 4.8 4.56 Meets Elocution 4.4 4.5 4.8 4.38 Meets

10/2014 15

MBA Program Assessment Results (2015-2018) SLO #4: Work Effectively in a Team Setting

Criterion 2015 2016 2017 2018 Assessment

Results Our Group Worked as a Team 5.0 4.6 4.6 4.80 Meets We Discussed Group Norms 3.9 4.3 3.6 4.40 Meets We Had an Initial Meeting to Get Organized

5.0 4.3 4.2 4.53 Meets

We Assigned Individual Responsibilities

5.0 4.8 4.5 4.80 Meets

Every Group Member Did Their Share of Work

4.8 4.4 4.5 4.87 Meets

Group Members Fulfilled Their Responsibilities

4.8 4.6 4.5 4.87 Meets

Group Members Were Prepared for Meetings

4.7 4.4 4.2 4.73 Meets

Group Member Contributions Met My Expectations

4.9 4.4 4.4 4.73 Meets

The Final Presentation Met My Expectations

4.8 4.8 4.5 4.73 Meets

I Could Not Do a Better Job If This Were an Individual Assignment

4.9 4.4 3.4 4.53 Meets

MBA Program Assessment Results (2015-2018) SLO #5: Apply Business Concepts to Managerial Decision Making

Criterion 2015 2016 2017 2018 Assessment

Results Identifies & Summarizes Problem at Issue

3.9 NA 4.0 4.50 Meets

Considers Stakeholders 3.9 NA 3.3 4.50 Meets Evaluation 3.9 NA 3.7 4.50 Meets Application of Analysis to Management Situation

3.9 NA 3.5 4.33 Meets

Chooses an Action 3.9 NA 3.2 4.33 Meets

10/2014 16

Appendix 1

10/2014 17

Appendix 2

10/2014 18

Appendix 3

10/2014 19

Appendix 4