SOUTHEASTERN OKLAHOMA STATE In\fIVERSITY

Department OfManagement and Marketing Program Outcome Assessment Report

General Business Program AY 2008-2009

September 11, 2009

Dt:3e;t:(---- ­Department ofManagement and Marketing

sey School of Business

1

Table of Contents

Page JMSB Mission Statement 3 Department Mission Statement 3 Program: General Business 3 AACSB Accreditation and Its Impact on the JMSB Assessment Process 3 Table 1. Learning Goals, Outcomes, Assessment Methods and Results 5 External Assessment Results 6 Figure 1. Overall Average Score for the JMSB vs. MFT Norm 6 Figure 2. General Business Major Scores 7 Analysis - Strengths/Weaknesses 7 Program Modifications 8 Web-based /IETV Additional Assessment 8 Figure 3. MFT Scores vs. Number of Online Courses 9 Faculty Contribution 9

2

JMSB Mission Statement

The mission of the John Massey School of Business is to be the premier regional business program in southeast Oklahoma and north Texas. We will prepare our business and aerospace students to meet the challenges in a competitive and ever changing economy by providing an excellent and continuously improving learning environment that emphasizes both academic and applied knowledge and research. We will further assist in the economic development of the region by providing effective application of knowledge with small and midsize businesses and Native American Tribes.

Department of Management and Marketing Mission Statement

The primary mission of the Department of Management and Marketing is to provide baccalaureate programs in Management, Marketing and General Business designed to provide an environment of academic excellence in undergraduate business education, and experiences both academic and applied that prepare students to operate in a diverse and global environment. Through these programs, students will develop an interest in lifelong learning.

The General Business Program

The General Business program is designed to prepare students for employment in a wide range of administrative positions in private business, government, or not-for-profit businesses. It is an interdepartmental major intended for students who wish to remain generalists rather than specialists in the business field. The curriculum promotes a general understanding of the business environment while providing a strong foundation in the common body of business knowledge. It includes required upper-division course work in accounting, finance, management and marketing, beyond the business core.

AACSB Accreditation and Its Impact on the JMSB Assessment Process

Last academic year, the JMSB earned AACSB accreditation. As part of the evaluation, AACSB sent a site visit team in October 2008. Following the visit, the three major recommendations from the team were to:

• Update the strategic plan • Redesign the Assurance of Learning (AoL) process to increase utility and efficiency • Revamp the MBA Admission and advising processes

As a result, the JMSB AoL Committee began redesigning the process. The JMSB AoL process had been in place since 2005; however, the site visit team suggested the process was too cumbersome and offered limited value for the effort required. Specifically, their recommendations were to (l) train faculty in AoL concepts and (2) redesign both the undergraduate and graduate current AoL processes to increase usefulness, while reducing faculty efforts in collecting data.

3

For (1), Dean David Martin from LaSalle University was hired in November 2008 to review the JMSB AoL process and deliver a seminar on how to improve the existing process. Additionally, AoL committee members attended the AACSB Assessment Conference held in Dallas in December 2008.

For (2), the AoL committee is currently redesigning the undergraduate AoL process as follows: • Reduce the ten current goals to a more manageable five. • Develop goal-specific rubrics for course-embedded assessment, to allow goal

statistics to be generated independent of faculty and course. • Develop an assessment plan that requires less data collection than the current process,

but requires faculty to address any deficiencies within a two-year planning cycle.

Table 1 shows the five new goals, objectives, learning outcomes, assessment method and results. Note that several goals are 'under development' or no data exists. The AoL committee plans to present its work to the Dean and faculty, as well as begin collecting course-specific data during the F09 semester. Once finalized, each program (management, marketing and general business) in the department will develop additional specific goals, objectives and traits to measure.

Consequently, no course-specific data was collected for AY0809. The only assessment data collected was the MFT in Business scores given to graduating seniors in the capstone course, MNGT 4633 Business Policy.

The MFT in Business is an objective, end-of-program test for the Business disciplines. The content of the Major Field Test in Business reflects the basic knowledge and understanding gained in the core undergraduate curriculum. The test is a two-hour, multiple-choice examination designed to assess mastery of concepts and principles as well as knowledge expected of students at the conclusion of a major in specific subject areas. It goes beyond measurement of factual knowledge, however, to evaluate students' abilities to analyze and solve problems, understand relationships, and interpret material. Students are also required to interpret graphs, diagrams, and charts related to the field of business. The test covers assessment indicators in nine areas: Accounting, Economics, Management, Quantitative Business Analysis, Finance, Marketing, Legal and Social Environment, Information Systems, and International Issues.

4

TabIlL. earnmg Goa s, outcomes, AssessmentMeth d sandResuItse 0 ASSESSMENT

RESULTSOBJECTIVES LEARNING OUTCOMES GOAlS METHODS

Cleu Logic & Organization

Effective use ofLanguagelStyle , ..Students will create well Correct Spelling and Gramrrar

written documents on a Course Embedded Assesstrenl (begin AY091O) Begin AY0910 Clear Purposebusiness topic Excellent Developtrent ofIdeas

Correct Citation and Documentation Goal 1. C01nmunication • School of Effective Organization business gradualeswill be effective

Knowledge ofContentb.StudenlS will deliver anconununicators with the ability to effective oral presentation Course Embedded ASSeSStrenl (begin AY091O) Begin AY0910Persuasive Conclusionprepare and deliver oral and written on a business topic Excellent delivery

technologies. presentations using appropriate

Effective use ofbody language

c.Students will usc Effective us ofMedia I. Beginappropriate technologies to ). Course Embedded Assessment (begin AY091 0)

AY0910enhance the effectiveness of 2. MIT Information Systems Subscore >= n01Ul

2. See Figures 1their written and oral trean (n =8) 3Effective Use of Technologypresentations.

Derme the problem

Research and gather Infonnation I. Begin

I. Couse Embedded Assesstrent (begin AY0910)Generate plans to sol ve problem, integratinga Use problem solving AY0910

relevant factors in decision making. 2. MFT Quantitati ve Methods Subscore >~ Dorm 2. See Figures 1trethod trean (n ~ 8)

3Select and implemenl best solution

Goals 2.J',til!lem solving - School Of Connnunicate the results [business gr~uales can apply critic8I Usc algebra to formulate/solve problems 1. Begin

J. Couse Embedded Assessment (begin AY09IO)thinking skills to prob1!'01 solving. correctlyb. Use quantitative AY0910 Use statistics methods to formulate/solvereasoning to solve problems 2. See Figures 12. MFT Quantitative Methods Subscore = norm problems correctly 3trean (n ~ 8)

c. Use technology such as specialized computer Use MS Ollice tools Sllch as Excel and

Couse Embedded Assessment (begin AY091 0) Begin AY0910ooftware to solve problems Access to solve problems correctly

Goals 3. Ethics· School ofbusiness graduates can identilY and assess under development ethicalissues in b·usiness .

. "" ~ ,

Goal 4; Leadership, Teamwork and Collllboration - SchocH ofBusiness graduates possesS th~leadeYllhip skills nece'¥'aI)I to aive dir~ctionand guidance,llt)d.toc!elegate wode tasks

under developmentin a·manner which proves·to .be eff~ctiveand which motivateScothers to do their best in both a manager/subordinate and a team sening,

Recognition oftlte critical importanceof Wlderstanding the concepts inherent in diversity issues.

GoalS. Divers~ . S'ehoolof Students will demonstrate an Application and detronstration ofindividualbusiness graduates willlIDderstand the appreciation ofhuman and understanding ofdiversity issues. I. Couse Embedded Assesstrent (begin AY0910)

diversity ofthe local an4global cultural differences through 2. MFf Legal and Social Environment Subscore >= See Figures 1- 3

Analyze, evaluate and assess the impact ofcOIJJIllunities, including cultural, a deeper understanding of norm trean (n =8)differences in individuals and groups.social, political and economic the world they inhabit. Evaluates, assesses, and rates the impact of college experience regarding diversity and its influence on personal and/or professional

difrerences.

lire

5

External Assessment Results

The Major Field Test Comparative Data (http://www.ets.org/Media/Tests/MFT/pdf/MFT%20PDFs%202007/Business4CMF.pdf) was used for comparison with the results achieved locally (see Appendix A for tables). The norm group represents over 83,323 senior students from 564 institutions who had taken the test between August 2006 and June 2008. Figure 1 shows the overall average score of the JMSB students compared to the norm since the fall 2002 semester. The minimum and maximum score is 120 and 200, respectively. Assessment indicators for the Business test cannot be compared to testing years prior to 2002 due to changes in the Business test that were introduced in 2002.

For the JMSB overall score (the diamond-shaped points), a 95% confidence interval is provided around the mean. Since the confidence interval contains the ETS overall average score (the square-shaped points), no significant difference exists between the JMSB average score and the overall average of the norm group. Although, no significance exists, the JMSB average score did drop from AY0708, and this was the first time that scores have dropped since 2005.

Figure 1. Overall Average Score for the JMSB vs. MFT Norm

_ ETS Overall Mean Score

-""'JMSB Overall Mean Score

In addition to the overall score, Figure 2 shows the average score for the nine assessment indicators, specific to General Business majors. The minimum and maximum score is 20 and 100, respectively. Similar to Figure 1, the General Business majors' average scores are represented with the diamond-shaped points and are bounded by a 95% confidence interval. Across the board, the subscores dramatically improved; however, one of the eight General

6

Business students who took the exam scored hi~her than any JMSB student since the exam has been given at SE. The student scored in the 99t percentile.

_ETS Comparative Mean

--2. Economics

80 -r-==""""""""""""""" 70

60

50

40

30

20 +-"----_+_

80....,......,,,,,,,...­

70

60

50

40

30

20

80 .-----..,......,-_,...-..,......,-...,

70

60

50

40

30

20

_ ETS Comparative Mean

--7. Legal and Social Environment

80 .--..,......,..,......,..,......,..,......,..,......,..,......,-..,......,..,......,...,

70

60

50

40

30

20

_ETS Comparative Mean

--4. Quantitative Business Analysis

80 1 ..,......,..,......,"7'7""""""""""""7'7"""""""':""'i'i'iT"'''1 70

60 ~.,.,.•.. ,'

50

40

30

20

80 .----,...---,...-,...-_,...--,

70

60

50

40

30

20

Figure 2. General Business Major Scores...~~~~ -~~~~ _ ETS Comparative Mean

-­ 1. Accounting

Analysis - StrengthslWeaknesses

A potential strength is the increased 'International Issues' subscore, which climbed to be significantly greater than the norm average. The General Business curriculum was modified two years ago, requiring an international course to be taken. Potentially, this modification may be linked to the increase.

7

Program Modifications

The brief calculus course, MATH 2143, was removed from the core because the course was not a prerequisite for any other course. In 2005, MATH 2143 was removed as a prerequisite for any BBA course. Although required for earning a BBA, MATH 2143 was not attached to any subsequent course. Some students delayed (are delaying) until their senior year. However, since 2005, standardized test scores (ETS Major Field Test in Business) have improved or remain unchanged both in finance and in quantitative methods.

Business calculus is not suggested by AACSB International. Several AACSB schools do not require the course. During our site visit in October, one of our reviewer's had recently removed business calculus as a required course from his business school.

Business calculus is taught infrequently at feeder schools even if technically on the books. Consequently, students must take when arriving at SE, which places these junior-level students in 2000-level courses. Grayson County CC requires a pre-calculus course prior to their brief calculus. The consequence is that to earn the A. S. at GCC within a 2 year period, the 2+2 agreement requires one semester of 19 hours.

Students can meet their general education math requirements through MATH 1513 College Algebra. For those students sufficient in college algebra, we will advise them to take a more advanced math course (such as brief calculus). We would also advise those planning to pursue an MBA to take brief calculus.

Web-based IIETV Additional Assessment

Within the JMSB, regression was used to measure the MFT overall score against the number of core online courses taken. The total number of core courses is 14. The data is for the last four years. Based on the data, Figure 3 shows that no relationship exists, and that MFT scores are not impacted by delivery method.

As course embedded assessment resumes within the JMSB, additional course-specific assessment will occur.

8

Figure 3. lVIFT Scores vs. Number of Online Courses Source: JMSB Seniors for the Last 4 Years with at least One Online Course

n= 136

y = -0.108x + 153.6 R2 = 0.000

~ 120 6 100 o 80 f 60 M 40

20 o

o 1 2 3 4

# Online Courses

5 6 7 8

200 180

F160

~ 140

• II .... ......•

Faculty Contribution

Dr. Jim Speers provided the online analysis. The AoL Committee (Drs. Speers, Campbell, Hrncir, Topuz) are revamping embedded course assessment. The department faculty reviewed and edited the report.

9

Appendix A - ETS Major Field Tests Comparative Data Tables

10

Listening. Learning. Leading.@M-aJor r/e'l~-'d Tests

Major Field Test in Business (form: 4CMF) Individual Students Total Score Distribution

Data includes Seniors from Domestic Institutions -- August 2006 to June 2008

Total Score % at or below

Range (120 - 200) 179 - 200 !lS

I 90

178 177 176 175 174 173 172

85 171 170 169 168 167

I 80166 165

75164 163 162 161 160

70

65

60

159 158 157 156 155 55154 153 152 45 151 150 149

40

35148 147 30146 145 25144 143 20142 141

15140 139 138

10137 136 135

5134 133 132

1?n _ 131

Number of Examinees 83323 Mean 152.9 Median 152 Standard Deviation 14.0

Total Scores are reported as scaled scores. % at or below based on percent below the lower limit

of the score interval.

Copyright © 2008 by Educational Testing Service. All lights reserved. ETS and the ETS logo are registered trademar1<s of Educational Testing Service

Listening. Learning. Leading.@M· ~. 'daJor rIe" Tests

Major Field Test in Business1 (form: 4CMF) Institutional Assessment Indicator Mean Score Distributions

Data includes Seniors from Domestic Institutions -- August 2006 to June 2008

Assessment Indicators2

Mean Percent % at or below

Mean Percent Correct Correct

AI1 AI2 AI3 AI4 AI5 AI6 AI7 AIS AI9 70 - 100 9 70 - 100

69 95 95 95 69 68 90 68 67 95 67 66 95 90 85 90 90 66 65 95 65 64 8fi 6485 80 9563 95 90 III 80 63 62 62 61 75 70 85 65 75 61 60 90

fifi 60 59 6t> t>u 65 59

I

58 90 Rr 'fi 58

57 80 5 55 0 35 60 57 56 fir

90 ?fi fifi 56

55 It> Il~ 9l 41 ;,Il t>l 55 54 Rn

85 85 lfi 4fi 54

53 65 75 35 30 0 10 40 53 52 ~[ ~fi 52 51 ;,It> ;,It>

5 ~l 51 50 ?[ ?fi 50 49 4 15 20 49 48 30 15 48 47 It> 47 46 ?fi

10 10 46 45 ;,Il

5 10 45 44 15 25 30 30 44 43 10

;,IU 't> 2;> 1

5 43 42 1fi 5 5 42 41 10 5 15 41 40 5 0 10 40 39 1 1 39 38 5 5 1 5 38 37 1 1 37

0-36 1 1 1 0-36

Number of Institution! 564 564 564 564 564 564 564 564 564 Mean 49.8 47.4 54.7 46.2 54.9 52.0 46.1 58.0 54.0 Median 50 47 55 46 55 52 46 58 54 Standard Deviation 6.9 7.0 7.5 5.9 8.5 6.8 6.1 5.2 8.0

'Assessment Indicators for this test cannot be compared to testing years prior to 2006 due to changes in the Business Test that were introduced in 2006.

2Assessment Indicator 1: Accounting Assessment Indicator 5: Finance Assessment Indicator 2: Economics Assessment Indicator 6: Marketing Assessment Indicator 3: Management Assessment Indicator 7: Legal and Social Environment Assessment Indicator 4: Quantitative Business Analysis Assessment Indicator 8: Information Systems

Assessment Indicator 9: International Issues

Assessment Indicators are reported as percent correct. % at or below based on percent below the lower limit of the score interval.

Copyright © 2008 by Educational Testing Service. All rights reserved. ETS and the ETS logo are registered trademarKs of Educational Testing Service.