University of Southern Indiana

Romain College of Business AACSB Sixth Year ReportUniversity of Southern IndianaBusiness

University of Southern IndianaRomain College of BusinessSixth Year Review ReportJanuary 15, 2014

Table of Contents

I: Introduction2

Issue Identified2

Expectations for resolution2

II: Responses to Issues Identified3

Concern #1: (a) The Apparent Lack of Maturity of the AOL system BS/BA program3

Program Learning Goals and Learning Objectives5

Learning Goal #1:5

Learning Goal #2:6

Learning Goal #3:6

Learning Goal #4:6

Learning Goal #5:6

Learning Goal #6:7

Learning Goal #7:7

Learning Goal #8:7

Performance Benchmarks for Program Learning Goals7

Outcomes Evidence for Program Learning Goals8

Program-Level Changes - Curricular and Assessment - Based on the AOL Process9

Explanation of Data Aggregation Using Rubrics10

Assessment Evidence: MBA Program Learning Goals (#2 -#9 and since Spring 2012 #2 - #5)17

Concern #2:20

Concern #3:21

III: Appendix24

University of Southern IndianaRomain College of Business

Sixth Year Review Report

I: Introduction

As noted in the letter from the Maintenance of Accreditation Committee, dated April 25, 2013, the Sixth Year Review period provides additional time to address a number of educational quality issues. This section highlights the issued identified and expectations for resolution based on the relevant accreditation standards identified in the April 25, 2013 letter.

Issue Identified

1. Concerns regarding the Assurance of Learning system across the BS/BA and MBA programs

a. It appeared that the AOL system was not fully matured

b. It appeared that the focus was on the assessment of majors instead of programs

Expectations for resolution

a. Provide documentation that clearly demonstrates that an AOL system is in place with learning goals and objectives across each program.

b. Be able to show that these learning goals have been measured using rubrics designed for assessing the activity

c. Be able to show that these learning goals are comprehensively evaluated

Where outcomes reveal deficiencies, curricular changes should be made

Issue Identified

2. Concern that the criteria for establishing and maintaining PQ status to ensure that professional experience and continual development activities are consistent with the teaching assignment.

a. It appeared that multiple PQ status faculty may not be meeting the requirements for maintenance of PQ status

Expectations for resolution

a. Review the criteria for establishing and maintaining PQ status to ensure that (a) professional experience and (b) continual development activities are consistent with the teaching assignment.

II: Responses to Issues Identified

Concern #1: (a) The Apparent Lack of Maturity of the AOL system BS/BA program

A number of steps were taken during the past ten months to address this concern. The first involved determination of the criteria for evaluating the maturity of the Colleges AOL system. Based on discussions, a review of the relevant literature, and knowledge sharing among faculty members, a five step framework based on a presentation at the 2013 International Conference and Annual Meeting[footnoteRef:1], was utilized at a faculty retreat on April 16, 2013 to evaluate the Colleges AOL system. These activities helped to develop a common understanding and following working definition of a mature AOL system. [1: Recasting Assurance of Learning by Kathryn Martell and Tracy Taylor at the April 2013 International Conference and Annual Meeting, Chicago. It was noted at the presentation that the framework was based on the 2007 (and subsequently updated in 2013) AACSB paper on Assurance of Learning Standards: An Interpretation. ]

A mature AOL system is one whose components are well established and function effectively to assure sustainability of the system. In particular, at the program level, there are clearly defined learning goals, with processes and activities that: (a) establish the extent of student learning; (b) serve as catalysts for curricula changes and other responses to learning outcomes evidence; and (c) sustain continuous improvements in curricula management and learning outcomes. In addition, these interdependent components serve to maintain stability of the AOL system while adapting to change resulting from internal and external influences.

Subsequent steps involved activities geared towards clarifying and modifying the Colleges AOL system, as well as implementation of the Colleges AOL plan. Issues identified during the Peer Review Team visit such as the lack of clarity about data aggregation of results pertaining to skill learning objectives and performance benchmarks for program-level data were also addressed in the process of clarifying the Colleges AOL system.

The major outcomes from the steps taken during the past ten months are:

A more precise statement of the number of program learning goals. There are eight program learning goals and previously, there were instances when seven program learning goals appeared to be identified. Sometimes the separate program goals, effective oral communication and effective written communication, were referred to as the effective communication program learning goal and in some instances the distinction between the skills-related learning goals and the knowledge-related program learning goal was the source of this tendency.

Outcomes for the program learning goals and the related performance benchmarks are now more clearly articulated. There is also clearer explanation and greater understanding of the process of aggregating assessment information for learning objectives to measure the learning outcomes for the program leaning goals.

There is now a clearer delineation of responsibilities between the Teaching Learning and Assessment Committee (TLAC) and the Deans office with regard to the collection, storage, analysis, recommendations, and dissemination of assessment information as well as the coordination of curriculum changes based on assessment information. Collection, compilation, and storage of assessment information for program learning goals are the responsibility of the Deans office. The TLAC is responsible for analysis, dissemination, recommendations, and coordination of curriculum changes.

The assessment plan has been modified so that assessment of all program learning goals is no longer conducted every semester. The modified assessment plan requires four program goals to be assessed every year. In 2013, the four program learning goals assessed were the knowledge-related program learning goal, and the teamwork, ethical decision-making, and analytical problem solving program learning goals. We are awaiting results for the major field test in business from the ETS conducted in Fall 2013 and the outcomes for the other three program learning outcomes are presented in the following section of the report. Assessment information for the prior years can be accessed at: http://business.usi.edu/aacsb/

The rubric associated with the technology-related program learning goal has been modified as a result of our AOL review during the past several months. (Appendix)

Awareness of a gap with regard to comprehensive assessment of the technology program goal at the end of the program has led to the decision to conduct the assessment of the technology program learning goal in the BS/BA programs capstone course. This will be implemented in conformity with the assessment plan during 2014.

Increased focus on reviewing the current program learning goals in the context of the themes of innovation, engagement, and impact with regard to continuous improvement contained in the 2013 business accreditation standards.

As indicated in Figure 1, processes and activities associated with each of the identified components have been in place since 2007 and the connections among these components have been enhanced since the Peer Review Teams visit. The maturity of the AOL system is reflected in the systematic and ongoing manner in which assessment information and insights provided about student learning have impacted curriculum management. In addition, the capacity of the AOL system to implement changes prompted by the Peer Review Teams visit and most recently, begin the process for incorporating the 2013 accreditation standards is reflective of its stability and effectiveness. Steps taken and a status report on the progress made toward a review of the business core curriculum are available at: http://business.usi.edu/aacsb/

Program Learning Goals and Learning Objectives

We have eight learning goals for the undergraduate BS/BA program. Each learning goal has associated learning objectives, which allow for the assessment of learning goals. What will students learn?

Learning Goal #1:

- Students will demonstrate a general understanding of the fundamental areas of business.

How will we know students demonstrate general understanding of the fundamental areas of business? Students will demonstrate a general understanding based on their performance in the Major Field Test in Business developed by the Educational Testing Service (ETS). Assessment information with regard to this program learning goal is obtained through administration of this test to undergraduate students in the Policy Formulation & Implementation course every Fall semester. The results of these tests are compared to the performance of other students across the country in the fo