Strategies for Implementing Program Level Assessment through Blackboard Outcomes
-
Upload
jeremy-anderson -
Category
Documents
-
view
56 -
download
2
Transcript of Strategies for Implementing Program Level Assessment through Blackboard Outcomes
![Page 1: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/1.jpg)
Strategies for Implementing Program-Level Assessment through OutcomesJeremy AndersonManager, Academic Computing
Dr. Kaitlin WalshInstructional Designer/Technologist
American International College
![Page 2: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/2.jpg)
About American International College
![Page 3: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/3.jpg)
Who are you?
• Faculty, staff, administration?
• What is your experience using Outcomes?– No experience– Heard of it– Investigated it – Adopted it
• Three burning questions?
![Page 4: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/4.jpg)
The Challenge
![Page 5: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/5.jpg)
Program Assessment Needs to Be:
• Systemic
• Sustainable
• And…
![Page 6: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/6.jpg)
Systematic
![Page 7: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/7.jpg)
Team-Based Approach
![Page 8: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/8.jpg)
Assessment at AIC
President
Provost
Institutional Effectiveness
Deans & Departments
Senate
Assessment Committee
Faculty Secretary*
EVP Administration
IT
![Page 9: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/9.jpg)
Working Across the Organization
Assessment Committee
Working Group
Pilot
![Page 10: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/10.jpg)
Methods to Scale - Preparation
![Page 11: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/11.jpg)
Getting the Framework in Place
External Feedback
• Assessment Plan Template
• Program Review Schedule
• Assessment Calendar
Internal Planning
• Other adopters
• Bb Consulting
![Page 12: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/12.jpg)
Assessment Plan Template
![Page 13: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/13.jpg)
Develop outcomes
Assessment Plan & Bb Outcomes
Gather Evidence &
Evaluate
Analyze & Discuss
Improve Instruction
Fall
SpringSum/Fall
Next AY
Assessment Calendar
![Page 14: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/14.jpg)
Institutional Support
• Assessment Day (2)
• Assessment Hour (1)
• ½ position to support Outcomes
![Page 15: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/15.jpg)
Methods to Scale - Adoption
![Page 16: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/16.jpg)
Questions for New Adopters – Readiness Check
1. Do you have an assessment lead?
2. Are your goals ready?
3. Are your assignments ready?
4. Are your rubrics ready?
5. What is your assessment calendar?a. Frequency of each goalb. Frequency of full cycle
![Page 17: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/17.jpg)
Questions for New Adopters – Operational Decisions
1. What is your collection period?
2. Who will complete evaluation sessions? How many must complete evaluations?
3. What sampling level will you require?
4. Will you need to keep samples of student work with your reports?
![Page 18: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/18.jpg)
Starting Small
Undergraduate Psych
• All outcomes
• 1 course
• Capstone assignment
• Independent evaluator
• Department rubric
MBA
• 2 outcomes
• 2 courses (1 o/c)
• Milestone assignments
• Independent evaluators
• AAC&U Value rubric
• Department rubric
![Page 19: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/19.jpg)
Training
• Introductory section for faculty and chairs in several prospective departments
• One-on-one training with individual assessment coordinators in pilot departments
• Reconvened for follow-up training
• Next step: documentation
![Page 20: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/20.jpg)
Roles – What Can They Do?
Alignments Goals Surveys RubricsEvidence
Collection
SystemAdministrator
AddRemove
CreateEdit
Delete
CreateEdit
Delete
CreateEdit
Delete
CreateEdit
Delete
AssessmentAdministrator Add
CreateEdit
Delete
CreateEdit
Delete
CreateEdit
Delete
CreateEdit
Delete
Assessment Manager None Run Reports None None
CreateEdit
Delete
RubricManager None Read-Only None
CreateEdit
DeleteNone
SurveyAuthor None Read-Only
CreateEdit
DeleteNone None
Goals Manager None
CreateEdit
DeleteNone None None
![Page 21: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/21.jpg)
Roles – What Can They See?
Admin Tab Outcomes TabOutcomes
DashboardSurveys
SystemAdministrator Yes Yes All All
AssessmentAdministrator Yes Yes All Only their own
Assessment Manager No Yes
Goals & AssessmentsOrganizations
None
RubricManager No No No No
SurveyAuthor No No No Only their own
Goals Manager Yes No No No
![Page 22: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/22.jpg)
Caveat – Class and Program Size
• AIC’s class and program sizes are generally very small, therefore no need to limit sample size when evaluating outcomes.– Also no real need to make
evaluations anonymous - instructor often involved in evaluation
• Small program sizes also impact outcomes planning– Limits to number of outside
evaluators and assessment coordinators
Click icon to add picture
Source: US News & World Report
![Page 23: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/23.jpg)
Things We Considered (But Didn’t Adopt Yet)
• Help Desk Ticket to submit Outcomes changes
• Organization to distribute reports
• Having faculty input own outcomes
![Page 24: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/24.jpg)
General Education Outcomes
![Page 25: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/25.jpg)
General Education at AIC
• Largely in flux
• Writing Intensive Courses
• Shared rubric for Title III
• AAC&U’s VALUE Rubrics
![Page 26: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/26.jpg)
Pilot
• COM2200 – Information & Technology
• Writing Intensive Course
• 4 sections, 3 faculty, 2 evaluators
![Page 27: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/27.jpg)
Next Steps
• Develop anchor set
• Train on the rubric
• Include other WICs
![Page 28: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/28.jpg)
Next Steps & Lessons Learned
![Page 29: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/29.jpg)
Building on What We Have
• New programs
• Assessment Hours throughout the year
• Expanding within departments
• ½-time position
![Page 30: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/30.jpg)
Lessons Learned – Standardize Everything!
![Page 31: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/31.jpg)
Lessons Learned – Rubric Design
![Page 32: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/32.jpg)
More on Reports
• Depending on the type of data you need, you may need to adjust your reports.– Convert rows to columns– Cleanup for SPSS– May be no problem for some faculty– Need different data? Submit an enhancement request
![Page 33: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/33.jpg)
Lessons Learned – Assignment Submission
• Needed to train some faculty on collecting assignments in Bb
• No direct submit
• No multiple attempts
• Group assignments not recommended
![Page 34: Strategies for Implementing Program Level Assessment through Blackboard Outcomes](https://reader034.fdocuments.us/reader034/viewer/2022042819/55cb370bbb61eb904e8b45d3/html5/thumbnails/34.jpg)
Unexpected benefit!
Using outcomes rubrics motivated faculty to explore the use of Blackboard’s rubric tool within their own courses.