Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the...
-
Upload
acbsp-global-accreditation -
Category
Business
-
view
33 -
download
0
Transcript of Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the...
![Page 1: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/1.jpg)
We’re Accredited! Now What?
Preparing the First QA Report
Presented by: Aaron Glassman, D.Mgt.Kelly Whealan George, PhD Dixie Button, DBA
![Page 2: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/2.jpg)
University Overview
• 3 campuses:
– Daytona Beach (~ 4500 UG & 500 grad)
– Prescott (~ 1500 students)
– World Wide (>150 locations, ~ 40 states, S. America, Europe, Asia and Middle East; ~26,000 FTE students)
• Emphasis is aviation & aerospace
• BS, MS and PhD programs
• ~ $17M in research
![Page 3: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/3.jpg)
Personnel Changes
Accreditation granted in December 2013. Since then . . .
• Third Reorganization
• Third Dean
• Third CAO
• Chancellor, Combined CAO/Chancellor, back to Chancellor
• New Accreditation campus liaison to work w/University liaison
• New faculty with faculty roles
• ACBSP co-champion leaves. New co-champion appointed 5 months before QA report due
![Page 4: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/4.jpg)
Personnel Changes
In the midst of personnel changes . . . .
• What is this thing called Notes and Conditions attached to the accreditation?
• What is this thing called a Quality Assurance (QA) report?
![Page 5: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/5.jpg)
Design Teamwork for Success
ACBSP Committee to address upcoming QA report issues
• Monthly teleconferences
• Knowledge management (Sharepoint)
• Developed a road map
• Attempt to intersect QA requirements with all other program requirements (SACS, QEP….)
• Spreads knowledge base around the College
![Page 6: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/6.jpg)
Conditions and Notes
Notes & Conditions attached to ACBSP accreditation
• Okay . . . But what does this mean?
• The initial accreditation letter says “Conditions and notes indicate that either the Standard or Criteria is not fully met, and should be viewed as an opportunity to move your program to a higher level of excellence.”
• First thought . . . Is satisfaction of Conditions & Notes due by the first QA report?
• No, just need to show that we are addressing them.
![Page 7: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/7.jpg)
Conditions and Notes
ACBSP Process Book - ConditionsMost standards are met, but there are some deficiencies in fully meeting all of the ACBSP standards.
ACBSP Process Book - NotesIn compliance with ACBSP standards, but not some of the criteria
. . . Notes represent concerns of the Board of Commissioners and until those concerns are dealt with by the institution and accepted by ACBSP, the progress on improving the process to remove the notes should be included in the quality assurance report submitted to ACBSP every two years.
![Page 8: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/8.jpg)
Conditions and Notes
Conditions must be removed by 9/15/2017 and progress on notes must be reported in future Quality Assurance Reports the first of which is due September 15, 2015
• Note on Standard 2
• Condition on Standard 3
• Condition on Standard 4, Criterion 4.2, 4.3, and 4.4
• Note on Standard 5, Criterion 5.3.2.b
• Note on Standard 6, Criterion 6.3.6 and 6.3.7
![Page 9: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/9.jpg)
Conditions and Notes
Note on Standard 2
• Referenced input from faculty in the development of both long-term and short-term strategic goals and objectives with measures, action plans, and timelines.
• Result: Removed
Condition on Standard 3
• Need feedback from stakeholders of the business programs other than students.
• Result: Condition reduced to a Note
![Page 10: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/10.jpg)
Conditions and Notes
Condition on Standard 4, Criterion 4.2, 4.3, and 4.4
• Show evidence of changes/improvements resulting from internal assessments
• Need to better document and analyze trend data as well as use comparative data to continuously improve student learning outcomes.
• Result: Conditions reduced to a Note
![Page 11: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/11.jpg)
Conditions and Notes
Note on Standard 5, Criterion 5.3.2.b
• Show progress on plan to ensure 40% of undergraduate credit hours taught by academically qualified faculty within two years.
• Result: Ongoing, Note maintained
Note on Standard 6, Criterion 6.3.6 and 6.3.7
• Document results & improvements resulting from newly implemented dashboard
• Result: Notes removed
![Page 12: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/12.jpg)
Shifting Priorities
• QEP (Undergraduate Research)
• Research
• Externally Funded Research
• Promotion and Tenure
• Cross-campus collaboration
![Page 13: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/13.jpg)
Publicly Available Information
ACBSP Requirement
• For Demonstrating Excellence in Baccalaureate/Graduate degree Business Programs the business unit must routinely provide reliable information to the public on its performance, including student achievement, such as assessment results.
![Page 14: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/14.jpg)
Publicly Available Information
What to include in publicly available information?
• Formed a committee, developed policy, execution ongoing. (http://worldwide.erau.edu/degrees-programs/colleges/business/index.html)
![Page 15: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/15.jpg)
Publicly Available InformationBiggest Challenges
Sources of Information
• ACBSP requirements from acbsp.org • Review of other ACBSP accredited colleges, exp. North
Carolina Central University and Gardner-Webb University
![Page 16: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/16.jpg)
Publicly Available InformationBiggest Challenges (Cont.)
What will be displayed
• Student Satisfaction Survey• Worldwide Alumni Survey• Graduation and Retention Rates for the College of Business• External Assessment Data (Peregrine), to fulfill the
requirement of publishing student learning outcome assessment results
![Page 17: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/17.jpg)
How we assess programs…..
• Courses have activities
• Activities are mapped to Learning Outcomes
• Learning Outcomes are mapped to Program Outcomes
• Program Outcomes Rubrics Embedded in LMS
• Learning Analytics Data Extracted from LMS
![Page 18: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/18.jpg)
Tips, Tricks, and Traps
Tip #1 – Do not map all activities even though it can be done.
• Only assess activities that have grade significance (students are less likely to skip the activity and grades are more normally distributed).
• Avoid mapping discussion board posts and other “route” weekly assignments as interrater reliability is usually poor even with a rubric.
![Page 19: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/19.jpg)
Tips, Tricks, and Traps
Tip #2 – Provide clear directions to faculty on using rubrics with LO assessments embedded.
• Faculty may unintentionally override the rubric to give the student a “0” or “100” score which means LO data is not collected.
• Faculty need to understand that attaining a LO does not reflect the students activity score. (e.g., student can earn a 75 in a paper for improper APA formatting and other non-content deductions but still achieve the mapped LO’s at the level of mastery)
![Page 20: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/20.jpg)
Tips, Tricks, and Traps
Trap #1 – We only think we are assessing programs.
• We are actually assessing course activities and the sum of the activities across multiple courses is how we gauge program health.
• Students from different programs may take the same course.
![Page 21: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/21.jpg)
Tips, Tricks, and Traps
Trap #2 – Binary/Administrative grades pollute the data.
• Students who receive zero scores for not submitting assignments skew learning outcome attainment data (averages) to the negative.
• Activities that are pass/fail (100pts) skew the outcome attainment data to the positive.
• These errors are hard to account for.
![Page 22: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/22.jpg)
Tips, Tricks, and Traps
Trick #1 – Use multiple data streams to assess programs.
• Direct Measure – Student Scores on Mapped Activity, Capstone, External Assessment, Artifact Grade/Regrade, etc. across multiple sections of multiple classes.
• Indirect Measure – Post-Course Feedback (Student), Post-Course Feedback (Faculty), Grade Distributions, Admissions vs. Graduation Rates
• Configure all systems with assessment in mind!
![Page 23: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/23.jpg)
Lessons Learned
• Fight for continuity
• Prepare yearly report similar to the 5th year QEP summary
• Incentivize key people
• Development by attending ACBSP conference
![Page 24: Kelly Whealan George, Aaron Glassman, and Dixie Button - We’re Accredited! Now What? Keeping the Work Moving to Prepare the QA Reports](https://reader031.fdocuments.us/reader031/viewer/2022030307/58eb68e11a28abbe1e8b4769/html5/thumbnails/24.jpg)
Contacts
Dr. Kelly Whealan George
Director of Accreditation, Assessment, and
Research Advancement of IgniteDiscipline Chair, Social Sciences
and Economics
Dr. Dixie ButtonMBA-A Program ChairACBSP Co-Champion
Dr. Aaron Glassman
Department Chair, Technology Management
Department