Online Program Review:

Reflections on Lessons Learned and Paving the Way Forward

Why we went online...

• Why’d you come?– What are the outcomes you’d like to achieve?– What do you want to learn?

• Guess what...– Our colleges had the same concerns

• In addition...

Introduction and Framework

• Antelope Valley College (Aeron)– Weave Online

• Cosumnes River College (Kathy)– Homegrown

• Mesa College (Bri & Jill)– TaskStream

• Yuba College (Erik)– TracDat

INTE

GRATED P

LANNIN

G

USING

ONLINE P

ROGRAM REVIE

W

AN

TE

L OP

E V

AL L E

Y C

OL L E

GE

BENEFITS OF GOING ONLINE

Universal access to the multiple constituents in comparison to a document.

Standard formatting and layout. Simple compliance reporting.

WEAVEONLINE

Customizable to effectively accommodate our PR questions.

Easy compliance reporting. Evidence database allows users to support

statements by attaching documentation to the questions.

No internal IT support needed Streamline reporting via PDF or MS Word

formats.

END USER EXPERIENCE

Common themes:

Likes Universal access

Simple entry process

Evidence database

Challenges Data overload

Unable to enter tables into document

No auto saving

CUSTOMIZATION

Customizable homepage which allows for news , updates and deadlines to be posted .

Question customizations from character limitations to link integration to HTML coding capabilities.

Customizable reporting.

WORKSPACE

INTEGRATION INTO OTHER PROCESSES

Same tool provides access to SLO data which can be essential in providing support in decisions related to action plans within the program review process.

Customized reports can focus on specific questions which can be used as evidence in other planning reports (e.g., Educational Master Plan, Staffing plans, ACCJC reports, etc.).

Cosumnes River College A Homegrown Online

Program Review System (PrOF)

Why Homegrown?Faculty driven project

VisionUnderstandingProgramming expertise

Why Homegrown?Practical Considerations

Commitment to current program review model Had immediate access to reassigned timeCould create connections with other databases No budget for ongoing costsCould institutionalize and expandCould phase in the roll-out

The questions in the Midterm PrOF were clear.

The online environment was easy to navigate.

The training I received was sufficient.

The Midterm PrOF contributed to reflection about our program.

The Midterm PrOF helped us plan for the future.

The Midterm PrOF reminded us of things we had forgotten.

The Midterm PrOF reminded us of our ac-complishments.

The Direction Tab helped me.

It was helpful to access other PrOFs

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Please indicate the degree to which you agree with the following statements (1 = Strongly Agree, 3 = Neutral, 5 = Strongly Disagree)

Selected CommentsI am typically the type of person that finds this

kind of activity frustrating, and a complete waste of my time. I feel PrOF has become a useful tool for real program evaluation as opposed to useless paperwork.

Filling out the sections really only took about two hours....the other hours were meeting with the fellow colleagues in the department to have a genuine program review.

Thanks for making it a smoother process for end users.

Can easily customize during implementation

Currentlyupdating and modifying PrOFproviding training to institutionalize and

fully utilize planning for development of other

applications

Customization

Information is easily extracted to inform planning and resource allocation fulfill other functions live and static extraction capabilities

System can be integrated with other systems to enhance integrity and save work

Need More Info?Dr. Katherine McLain, Dean of College

Planning and Research; mclaink@crc.losrios.edu; 916-691-7144

Mark Ford, Faculty Developer and Librarian fordm@crc.losrios.edu; 916-691-7628

INTEGRATED PLANNING USING ONLINE PROGRAM REVIEW

RP ConferenceApril 1, 2013

WHY TASKSTREAM? Worked with our program review model

Could handle our complex process (e.g., multiple reviewers, collaboration with Liaisons, etc.)

Cost-effective College was already using TaskStream for SLO

assessment Can integrate with SLO component

Customization capabilities Workspaces, forms, and roles/access are created,

modified, or managed locally Reporting capabilities

Real-time analytics and Excel exporting capabilities

THE USER EXPERIENCE

Online module was tested and adjusted based on feedback from the Program Review Committee and shared governance groups

What did users like? 24/7 access Virtual spaces for collaboration One-stop shop for data, program review, resource

requests What did users find challenging?

The software learning curve Managing multiple roles (Lead Writer, Liaison, etc.) Reviewer functions

Based on preliminary feedback in fall 2012…Process evaluation in progress…

CUSTOMIZING IT FOR MESA

Customized workspace template that can be adjusted with some minor programming

Workspace layouts, data, attachments, and requirements were adjusted to fit college needs

Custom forms/fields and attachment capabilities

Created custom feedback forms for reviewers Ability to create custom rubrics in future

Exported reports to Excel for reformatting and customization

WHAT IT LOOKS LIKE: WORKSPACE LAYOUT

WHAT IT LOOKS LIKE: INSTRUCTION UPDATE FORM

WHAT IT LOOKS LIKE: NEW GOALS FORM

PULLING IT ALL TOGETHER

Online module allows for a more dynamic experience for users

Information is stored and can be viewed or exported as: A holistic report (full program review

document) OR As pieces (e.g., new goals, resource

requests, etc.) to inform planning and resource allocation recommendations

REPORTINGAggregate

Reports

Individual

Program

Reports

QUESTIONS?

Jill Baker, Dean of Institutional Effectiveness jibaker@sdccd.edu 619-388-2320

Bri Hays, Campus Based Researcher bhays@sdccd.edu 619-388-2319

Erik Cooper

ecooper@yccd.edu

Why TracDat?

Using TracDat for SLOs› WCC Faculty› Considered home grown

Adobe Database Survey Monkey Colleague Integration

The User Experience

“TracDat: An ancient Swedish word meaning ‘evil one’”

Year 1› Unintuitive› Unfamiliar› Unexcited› Unsure

The User Experience

Year 2› Collaborative› Time Saving› Meaningful› Useful› Still Unintuitive...but more support helps

Administration

Integration with MIS› Excel uploads...so not really

Integration with Planning/Budgeting› Standard and Ad Hoc reports› Multi-user access...except

Customization› Multi-college, similar but different

processes› Easy enough

Reporting

Reporting

Lessons Learned

• Trials, Pains, and Refinement– Implementation• Successes• Challenges

– Evolution• Cultural considerations• Next steps

Questions?

Need to contact us?

• Aeron Zentner– azentner@avc.edu

• Bri Hays– bhays@sdccd.edu

• Jill Baker– jibaker@sdccd.edu

• Kathy McLain– mclaink@CRC.losrios.edu

• Erik Cooper– ecooper@yccd.edu