Annual Academic Assessment Report Cover Sheet · The MBA teaching faculty and MBA Programs office...
Transcript of Annual Academic Assessment Report Cover Sheet · The MBA teaching faculty and MBA Programs office...
10/2014 1
Annual Academic Assessment Report Cover Sheet Assessment reports are due the 1st Wednesday after the Fall Term
Email to: [email protected] Program Information:
Program Assessed
Master of Business Administration (MBA)
Department
MBA Programs Office
College
Lee Business School (LBS)
Department Chair Chin-Chun (Vincent) Hsu, Ph.D., CMA
Assessment Coordinator
Chin-Chun (Vincent) Hsu, Ph.D., CMA
Date Submitted December 31, 2018
Contact Person for This Report
Name Chin-Chun (Vincent) Hsu, Ph.D., CMA
Phone 895-3842
Please attach a narrative (not to exceed 4 pages, excluding appendices) addressing the following:
What are the student learning outcomes? Please provide a numbered list.
Which learning outcomes were assessed?
How were they assessed? (Programs must use at least one direct assessment of student learning.)
Undergraduate programs should assess at least one University Undergraduate Learning Outcome (UULO) each year, which may or may not overlap with a program learning outcome.
Graduate programs should assess at least one outcome related to one of the following graduate level requirements each year:
o student engagement in research, scholarship, creative expression and/or appropriate high-level professional practice.
o activities requiring originality, critical analysis and expertise. o the development of extensive knowledge in the field under study.
What was learned from the assessment results?
How did the program respond to what was learned?
Please limit the narrative portion of your report to no more than four pages. You may attach appendices with data, tables, charts, or other materials as needed. Please explain the relevant conclusions from any appendices in your narrative. Please contact the Office of Academic Assessment if you have questions or need assistance.
10/2014 2
What are the student learning outcomes? Please provide a numbered list.
A key aspect of the MBA Programs mission centers on helping students gain the necessary market-driven knowledge and professional skills to succeed in contemporary and future business environments. Toward this end, five learning goals spanning a broad range of knowledge and skill areas motivate the design and structure of the MBA programs. Specifically, upon completion of the MBA program, students should be able to:
1. SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business
decisions
2. SLO 2: Communicate effectively business ideas and analyses in writing
3. SLO 3: Communicate effectively business ideas and analyses in oral presentations
4. SLO 4: Work effectively in a team setting
5. SLO 5: Apply business concepts to managerial decision making
Which learning outcomes were assessed?
Annually, the MBA Programs Office completes the following assessment process for each student learning outcome.
1. Collect course artifacts from appropriate MBA curriculum courses
2. Assess performance of course artifacts against learning goal performance standards
3. Identify learning goal areas of sub-par performance
4. Recommend actions to raise performance in sub-par areas to par or higher levels.
5. Present results and recommendations to MBA faculty. In 2018, we did the assessment by following the curriculum map in the newly developed 2015-2018 Three-year Assessment Plan. In 2018, we assessed all SLOs through several groups of students:
We assessed SLO 1 through MBA 787
We assessed SLO 2 through MBA 763 and MBA 767.
We assessed SLO 3 through MBA 787.
We assessed SLO 4 through MBA 787.
We assessed SLO 5 through MBA 787.
How were they assessed? (Programs must use at least one direct assessment of student learning.)
The methodology relies heavily on course-embedded assessment instruments to generate course artifacts reflective of program progress toward the learning goals. Utilizing the course-
10/2014 3
embedded system of assessment will allow for detailed outcome-level data on achievement of the outcomes connected to each program goal. The MBA teaching faculty and MBA Programs office developed rubrics and assessment data collection forms to assess student achievement of the learning outcomes. Please refer to Appendix 1-5 for sample rubrics and sample forms. The forms identify the performance levels in the following fashion – Level 5 (Excellent), Level 4 (Good), Level 3 (Adequate), Level 2 (Weak), and Level 1 (Poor). We assessed SLO 1 through core course MBA 787 [Direct Assessment: course-embedded assessment].
MBA students will recognize and evaluate ethical, legal and regulatory dimensions of
business decisions.
Written ethics case analyses in MBA 787 (Capstone: Strategic Management) rated
against 5 criteria on 5-point scale: (1) Identification of Ethical Issues, (2) Consideration of
Stakeholders, (3) Analysis of Issue from Multiple Ethical Perspectives, (4) Discussion of
Options, and (5) Proposal of Decision. [See Appendix 1].
We assessed SLO 2 through core course MBA 763 and MBA 767 [Direct Assessment: course-embedded assessment].
MBA students will demonstrate ability of communicating effectively business ideas and
analyses in writing.
Written case analyses in MBA 763 (Leadership and Team) and MBA 767 (Marketing
Opportunity Analysis) rated against 4 writing criteria on 5-point scale: (1) Clarity &
Conciseness, (2) Professional Tone, (3) Organization, and (4) Grammar & Mechanics.
[See Appendix 2].
We assessed SLO 3 through core course MBA 787 [Direct Assessment: course-embedded assessment].
MBA students will demonstrate ability of communicating effectively business ideas and
analyses in oral presentation.
20-30 minute oral presentations of group project results in MBA 787 (Capstone:
Strategic Management) rated against 6 verbal communication criteria on 5-point scale:
(1) Organization, (2) Subject Knowledge, (3) Graphics, (4) Mechanics, (5) Eye Contact,
and (6) Elocution. [See Appendix 2].
10/2014 4
We assessed SLO 4 through core course MBA 787 [Indirect Assessment].
SLO 4 is to measure the student’s ability to work effectively in a team setting.
Students in MBA 787 (Capstone: Strategic Management) complete a 10-item
“Group Processes and Peer Evaluation Forms” questionnaire reflective of group
and individual-level facets of team work. (See Appendix 4).
This form was intended to measure the degree of effectiveness of the groups and the
extent to which all members did their share of the work.
We assessed SLO 5 through core course MBA 787 [Direct Assessment: course-embedded assessment].
SLO 5 is to measure the student’s ability to apply business concepts to managerial
decision making.
As the culminating experience for all Lee Business School’s MBA students, MBA 787
Capstone Business Projects give students the opportunity to demonstrate mastery of
the skills and knowledge gained throughout their program.
Project teams are made up of 4-5 students who are nearing the end of their program.
Teams contribute 600 to 900 hours of research and consulting work on the project,
working closely with the client company and their faculty advisor. Teams provide the
client with a presentation of their findings and recommendations. Presentations of
capstone business projects for client companies in MBA 787 rated by 5-10 Las Vegas
business executives and several graduate faculty members. (See Appendix 5).
Undergraduate programs should assess at least one University Undergraduate Learning Outcome (UULO) each year, which may or may not overlap with a program learning outcome.
Not Applicable. The Master of Business Administration (MBA) degree is a business graduate program. UULO does not apply to graduate programs.
Graduate programs should assess at least one outcome related to one of the following graduate level requirements each year: o student engagement in research, scholarship, creative expression and/or appropriate high-level
professional practice. o activities requiring originality, critical analysis and expertise. o the development of extensive knowledge in the field under study
The Master of Business Administration (MBA) degree is a non-thesis program designed to serve students, alumni, and businesses by providing educational programs that enhance leadership and competence among business people, especially business practitioners, and provides solutions for important business problems.
10/2014 5
What was learned from the assessment results?
We assessed SLO 1 through capstone course MBA 787 Results:
1. Table 1 and Table 2 report the results from MBA 787 Strategic Management. The SLO 1
were assessed using assessment rubric with five-point scale. As shown in Table 1, the
median scores for all five dimensions are in 4 or better.
2. Table 2 shows the percentage of students meeting and exceeding expectation. In terms
of Identification of Ethical Issues, most students do very well; 100% of students have
reached “Good” or “Excellent” levels. In terms of Consideration of Stakeholders, most
students do well with 100% of students have reached “Good” or “Excellent” levels. In
terms of Analysis of Issue from Multiple Ethical Perspectives, 100% of students have
reached “Good” or “Excellent” levels. In terms of Discussion of Options, 100% of
students have reached “Good” or “Excellent” levels. In terms of Proposal of Decision,
100% of students have reached “Good” or “Excellent” levels.
Table 1 Summary Statistics
SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business decisions
Rubric Mean Median St.Dev. Min Max N
Identification of Ethical Issues 4.17 4 0.41 4 5 16
Consideration of Stakeholders 4.83 4 0.41 4 5 16
Analysis of Issue from Multiple Ethical Perspectives 4.33 4 0.52 4 5 16
Discussion of Options 4.50 4 0.55 4 5 16
Proposal of Decision 4.33 4 0.52 4 5 16
10/2014 6
Table 2 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 1: Recognize and evaluate ethical, legal and regulatory dimensions of business decisions
Rubric Excellent
[5]
Good
[4]
Adequate
[3]
Weak
[2]
Poor
[1]
Identification of Ethical Issues 17 % 83 % 0 % 0 % 0 %
Consideration of Stakeholders 83 % 17 % 0 % 0 % 0 %
Analysis of Issue from Multiple Ethical Perspectives 34 % 66 % 0 % 0 % 0 %
Discussion of Options 50 % 50 % 0 % 0 % 0 %
Proposal of Decision 34 % 66 % 0 % 0 % 0 %
We assessed SLO 2 through core course MBA 763 and MBA 767. Results:
1. Table 3 and Table 4 report the results from MBA 763 Leadership, Teams and Individuals.
The SLO 2 were assessed using assessment rubric with five-point scale.
2. Table 4 shows the percentage of students meeting and exceeding expectation. In terms
of Clarity and Conciseness, most students do very well; 85% of students have reached
“Good” or “Excellent” levels. In terms of Professional Tone, 85% of students have
reached “Good” or “Excellent” levels. In terms of Organization, 80% of students have
reached “Good” or “Excellent” levels. In terms of Grammar and Mechanics 85% of
students have reached “Good” or “Excellent” levels.
Table 3 Summary Statistics
SLO 2: Communicate effectively business ideas and analyses in writing
Rubric Mean Median St.Dev. Min Max N
Clarity And Conciseness 4.21 4 .70 3 5 39
Professional Tone 4.21 4 .70 3 5 39
Organization 4.08 4 .77 2 5 39
Grammar and Mechanics 4.15 4 .81 2 5 39
10/2014 7
Table 4 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 2: Communicate effectively business ideas and analyses in writing
Rubric Excellent
[5]
Good
[4]
Adequate
[3]
Weak
[2]
Poor
[1]
Clarity And Conciseness 36 % 49 % 15 % 0 % 0 %
Professional Tone 36 % 49 % 15 % 0 % 0 %
Organization 31 % 49 % 17 % 3 % 0 %
Grammar and Mechanics 36 % 49 % 10 % 5 % 0 %
Results:
1. Table 5 and Table 6 report the results from MBA 767 Market Opportunity Analysis. The
SLO 2 were assessed using assessment rubric with five-point scale. As shown in Table 5,
the median scores for all five dimensions are in 4.
2. Table 6 shows the percentage of students meeting and exceeding expectation. In terms
of Clarity and Conciseness, most students do very well with a median of 4.0; 77% of
students have reached “Good” or “Excellent” levels. In terms of Professional Tone, most
students do well with a median score of 4 with 93% of students have reached “Good” or
“Excellent” levels. In terms of Organization, the median score is 4.0 and 53% of students
have reached “Good” or “Excellent” levels. In terms of Grammar and Mechanics, the
median score is 4.0 and 69% of students have reached “Good” or “Excellent” levels.
Table 5 Summary Statistics
SLO 2: Communicate effectively business ideas and analyses in writing
Rubric Mean Median St.Dev. Min Max N
Clarity And Conciseness 3.85 4 .15 3 5 13
Professional Tone 4.25 4 .18 3 5 13
Organization 3.69 4 .21 3 5 13
Grammar and Mechanics 3.69 4 .13 3 4 13
10/2014 8
Table 6 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 2: Communicate effectively business ideas and analyses in writing
Rubric Excellent
[5]
Good
[4]
Adequate
[3]
Weak
[2]
Poor
[1]
Clarity And Conciseness 8 % 69 % 23 % 0 % 0 %
Professional Tone 31 % 62 % 7 % 0 % 0 %
Organization 15 % 38 % 47 % 0 % 0 %
Grammar and Mechanics 0 % 69 % 31 % 0 % 0 %
We assessed SLO 3 through core course MBA 787. Results:
1. Table 7 and Table 8 report the results from MBA 787 Accounting for Managers. The
SLO 3 were assessed using assessment rubric with five-point scale. As shown in Table 7,
the median scores for all six dimensions are in 4 or better.
2. Table 8 shows the percentage of students meeting and exceeding expectation. In terms
of Organization, most students do very well with a median of 5.0; 95% of students have
reached “Good” or “Excellent” levels. In terms of Subject Knowledge, most students do
well with a median score of 5.0 with 100% of students have reached “Good” or
“Excellent” levels. In terms of Graphics, the median score is 5.0 and 92% of students
have reached “Good” or “Excellent” levels. In terms of Mechanics, the median score is
5.0 and 100% of students have reached “Good” or “Excellent” levels. In terms of Eye
Contact, the median score is 4.0 and 77% of students have reached “Good” or
“Excellent” levels. In terms of Elocution, the median score is 5.0 and 96% of students
have reached “Good” or “Excellent” levels.
10/2014 9
Table 7 Summary Statistics
SLO 3: Communicate effectively business ideas and analyses in oral communication
Rubric Mean Median St.Dev. Min Max N
Organization 4.63 5 0.50 4 5 16
Subject Knowledge 4.44 5 0.51 4 5 16
Graphics 4.38 5 0.62 4 5 16
Mechanics 4.44 5 0.51 4 5 16
Eye Contact 4.56 5 0.51 4 5 16
Elocution 4.38 5 0.50 4 5 16
Table 8 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 3: Communicate effectively business ideas and analyses in oral communication
Rubric Excellent
[5]
Good
[4]
Adequate
[3]
Weak
[2]
Poor
[1]
Organization 63 % 37 % 0 % 0 % 0 %
Subject Knowledge 44 % 56 % 0 % 0 % 0 %
Graphics 44 % 49 % 7 % 0 % 0 %
Mechanics 44 % 56 % 0 % 0 % 0 %
Eye Contact 56 % 44 % 0 % 0 % 0 %
Elocution 38 % 62 % 0 % 0 % 0 %
10/2014 10
We assessed SLO 4 through core course MBA 787. Results:
1. Table 9 and Table 10 report the results from MBA 787 Capstone Project assessment.
The SLO 4 were assessed using assessment rubric with five-point scale (5 = strongly
agree and 1 = strongly disagree).
2. The students were asked to rate their team’s performance. The percentage of students
rating their team as “strongly agree” or “agree” for each question is shown in Table 10.
It should be noted that the results represent all students who completed ratings for
each question. As shown in Table 9, the median scores for all 10 questions are in 5.
Table 9 Summary Statistics
SLO 4: Wok effectively in a team setting
Mean Median St.Dev. Min Max N
Our group worked as a team. 4.80 5 0.77 2 5 15
We discuss group norms. 4.40 5 0.91 2 5 15
We had an initial group meeting get organized. 4.53 5 1.06 1 5 15
We assign individual responsibilities. 4.80 5 0.77 2 5 15
Every group member did their share of the work. 4.87 5 0.35 4 5 15
Group members fulfilled their responsibilities as assigned. 4.87 5 0.52 3 5 15
Group members were prepared for group meetings. 4.73 5 0.80 2 5 15
Contributions of group members met my expectations. 4.73 5 1.03 1 5 15
The final presentation met my expectations. 4.73 5 1.03 1 5 15
I could not do a better job if this were an individual assignment. 4.53 5 1.13 1 5 15
10/2014 11
Table 10 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 4: Work effectively in a team setting
Strong Agree
[5]
Agree
[4]
Cannot Decide
[3]
Disagree
[2]
Strongly Disagree
[1]
Our group worked as a team. 93 % 0 % 0 % 7 % 0 %
We discuss group norms. 60 % 26 % 7 % 7 % 0 %
We had an initial group meeting get organized. 73 % 20 % 0 % 0 % 7 %
We assign individual responsibilities. 93 % 0 % 0 % 7 % 0 %
Every group member did their share of the work. 87 % 13 % 0 % 0 % 0 %
Group members fulfilled their responsibilities as assigned. 93 % 0 % 7 % 0 % 0 %
Group members were prepared for group meetings. 87 % 7 % 0 % 7 % 0 %
Contributions of group members met my expectations. 93 % 0 % 0 % 0 % 7 %
The final presentation met my expectations. 93 % 0 % 0 % 0 % 7 %
I could not do a better job if this were an individual assignment. 80 % 7 % 7 % 0 % 7 %
10/2014 12
We assessed SLO 5 through core course MBA 787. Results:
1. Table 11 and Table 12 report the results from MBA 787 Capstone Project assessment.
The SLO 5 were assessed using assessment rubric with five-point scale (5 = strongly
agree and 1 = strongly disagree).
Table 11 Summary Statistics
SLO 5: Apply business concepts to managerial decision making
Rubric Mean Median St.Dev. Min Max N
Identifies & Summarizes Problem at Issue 4.50 5 0.55 4 5 16
Considers Stakeholders 4.50 5 0.55 4 5 16
Evaluation 4.50 5 0.55 4 5 16
Application of Analysis to Management Situation 4.33 5 0.52 4 5 16
Chooses an Action 4.33 5 0.52 4 5 16
Table 12 Assurance of Learning
(Percentage of Students Achieving Learning Goals)
SLO 5: Apply business concepts to managerial decision making
Rubric Excellent
[5]
Good
[4]
Adequate
[3]
Weak
[2]
Poor
[1]
Identifies & Summarizes Problem at Issue 40 % 60 % 0 % 0 % 0 %
Considers Stakeholders 60 % 40 % 0 % 0 % 0 %
Evaluation 20 % 80 % 0 % 0 % 0 %
Application of Analysis to Management Situation 20 % 80 % 0 % 0 % 0 %
Chooses an Action 60 % 40 % 0 % 0 % 0 %
10/2014 13
How did the program respond to what was learned?
1. The MBA assessment (assurance of learning) process is valuable for learning what areas of
improvement are needed to ensure MBA students are learning what we believe they are
learning. The assessment is performed using course-embedded techniques.
2. Utilizing the course-embedded system of assessment will allow for detailed outcome-level
data on achievement of the outcomes connected to each program goal. The MBA
Assessment Committee develops assessment forms and rubrics, and the faculty members
teaching each of the courses identified in the Curriculum map incorporate assessment tasks
into their courses. All outcomes are assessed in required courses identified in the
Curriculum Map.
3. The program-level benchmark for Excellence is set for 100 percent of students that
completed the MBA program with a passing assessment to achieve Level 5 [Excellence] or
Level 4 [Good] performance in all SLOs. The program-level benchmark for satisfactory
performance is set for 80 percent of students that completed the course with a passing
assessment to achieve Level 5 [Excellence] or Level 4 [Good] performance in all SLOs.
4. Overall speaking, we are meeting goal however greater emphasis on business decision-
making (SLO 5) is desired in the program. The faculty are currently considering ways to
improve the assessment of this learning goal in the overall program.
5. Summaries of assessment results from 2015 to 2018 are presented here. Overall assessment
Scheme: “Fails to Meet Criterion” if Mean Rating = 1 or 2, “Marginally Meets Criterion” if
Mean Rating >2 and <4, “Meets Criterion” if Mean Rating = 4 or 5.
10/2014 14
MBA Program Assessment Results (2015-2018) SLO #1: Recognize & Evaluate Ethical, Legal, & Regulatory Dimensions of Business Decisions
Criterion 2015 2016 2017 2018 Assessment
Results
Identification of Ethical Issues 2.7 3.8 3.4 4.17 Meets
Consideration of Stakeholders 2.7 4.1 3.4 4.83 Meets
Analysis of Issue from Multiple Ethical Perspectives
2.7 4.0 4.0 4.33 Meets
Discussion of Options 2.7 4.0 3.3 4.50 Meets
Proposal of Decision 2.7 4.3 3.0 4.33 Meets
MBA Program Assessment Results (2015-2018) SLO #2 Communicate Effectively Business Ideas & Analyses in Writing
Criterion 2015 2016 2017 2018 Assessment
Results Clarity & Conciseness 4.4 3.7 4.2 4.21 Meets Professional Tone 4.4 4.1 4.1 4.21 Meets Organization 4.4 3.9 4.6 4.08 Meets Grammar & Mechanics 4.4 3.8 4.3 4.15 Meets
MBA Program Assessment Results (2015-2018) SLO #3: Communicate Effectively Business Ideas & Analyses in Oral Presentation
Criterion 2015 2016 2017 2018 Assessment
Results Organization 4.4 4.7 4.6 4.63 Meets Subject Knowledge 4.4 4.7 4.4 4.44 Meets Graphics 4.4 4.6 NA 4.38 Meets Mechanics 4.4 5.0 4.9 4.44 Meets Eye Contact 4.4 4.0 4.8 4.56 Meets Elocution 4.4 4.5 4.8 4.38 Meets
10/2014 15
MBA Program Assessment Results (2015-2018) SLO #4: Work Effectively in a Team Setting
Criterion 2015 2016 2017 2018 Assessment
Results Our Group Worked as a Team 5.0 4.6 4.6 4.80 Meets We Discussed Group Norms 3.9 4.3 3.6 4.40 Meets We Had an Initial Meeting to Get Organized
5.0 4.3 4.2 4.53 Meets
We Assigned Individual Responsibilities
5.0 4.8 4.5 4.80 Meets
Every Group Member Did Their Share of Work
4.8 4.4 4.5 4.87 Meets
Group Members Fulfilled Their Responsibilities
4.8 4.6 4.5 4.87 Meets
Group Members Were Prepared for Meetings
4.7 4.4 4.2 4.73 Meets
Group Member Contributions Met My Expectations
4.9 4.4 4.4 4.73 Meets
The Final Presentation Met My Expectations
4.8 4.8 4.5 4.73 Meets
I Could Not Do a Better Job If This Were an Individual Assignment
4.9 4.4 3.4 4.53 Meets
MBA Program Assessment Results (2015-2018) SLO #5: Apply Business Concepts to Managerial Decision Making
Criterion 2015 2016 2017 2018 Assessment
Results Identifies & Summarizes Problem at Issue
3.9 NA 4.0 4.50 Meets
Considers Stakeholders 3.9 NA 3.3 4.50 Meets Evaluation 3.9 NA 3.7 4.50 Meets Application of Analysis to Management Situation
3.9 NA 3.5 4.33 Meets
Chooses an Action 3.9 NA 3.2 4.33 Meets