Post on 16-Jan-2016
Program Review, Service Unit Outcomes and Planning
Eva Bagg, Associate Dean of Institutional Effectiveness
Maria Narvaez, Planning Analyst
July 8, 2009
Agenda
• Overview• LBCC program planning & review• Timeline • Components of program planning & review• How to begin developing mission statement,
goals & SUOs• Purpose & principles of assessment• Unit assignment• Sneak preview of TracDat• Questions/Sign-up
Overview
• Today’s workshop will– Provide context for SUOs– Guidance to prepare content for program
planning and review• Customized Training sessions with each
administrative unit will follow today’s workshop– Sign up for a time that works for your unit
LBCC program planning & review:
a historical perspective
ProgramReview – every 3 yrs
Dept. Planning -No formal
process
SUOs
Non-Instructional
Dept. Planning -process
had problems
ProgramReview – every 6 yrs
SLOs, But no
assessment plans
Instructional
The new LBCC program planning & review process
Program Review Planning
Outcomes Assessment
Student Success &
Institutional Effectiveness
Timeline for first implementation of new
process• July – customized training sessions &
units draft SUOs• July/ August – feedback to units• August – hands-on support to enter
information into TracDat• Program Reviews are DUE on Oct. 1st *
• Program Plans/ Reviews submitted to Validation Teams and then to “inter-level” groups then to VP Level.
* We hope most Admin Units will have their program reviews completed by College Day.
Program planning & review
Mission Statement
• A unit mission statement should:– identify the unit’s purpose– reflect how the unit supports a student learning
environment – be distinctive– articulate the essential functions/activities of
the unit– identify the primary stakeholders of the unit– align with LBCC’s mission
• See handout for examples
Goals• Goals are:
– overarching general statements that describe the department's strategic direction.
– usually not measurable but rather, they represent coordinated efforts of intended improvements for the department’s processes
– what the department plans to do and are reflective of long-term priorities
– representative of consensus of staff aspirations for the program
– primarily used for general planning and are used as the starting point to the development and refinement of outcomes.
• See handout for examples
How to get started
(1) Meet with your unit to discuss and gain consensus on your unit mission– If your unit comprises multiple functional areas,
make sure each area is represented when you meet.
– Discuss the purpose of your unit (unit mission)• What are the primary functions or activities of your
unit?• Who are your stakeholders? Who do you provide
services to and/or who benefits from the services?
How to get started
(2) Meet with your unit to discuss and gain consensus on your unit goals– Review the goals stated in your last program
review.• How far have you come to achieving these goals? • Are they still relevant?
– Decide which previous goals your unit will retain (and/or refine).
– Write any new goals that state what you want to improve or modify about your unit in the next 3-5 years. (try to limit to 5)
What are Service Unit Outcomes?
LBCC’s definition of a SUO: Outcomes describe the desired quality
(timeliness, accuracy, responsiveness, etc.) of key functions and services within the support or administrative unit.
SUOs are expressed as statements of what users of the service experience, receive or understand as a result of a given service interaction. These should address both process and satisfaction indicators.
What are Service Unit Outcomes?
Types of SUOs
• Process outcomes– The level or volume of unit activity – The efficiency with which the unit’s processes are conducted – Compliance with external standards of “good practice in the field” or
regulations– Effectiveness of the service in accomplishing intended outcome
• Client learning can also be an outcome, but the results are used to improve the effectiveness of the unit
• Satisfaction outcomes– Satisfaction-oriented administrative objects range from overall
“client/customer satisfaction” as included on institutional graduating student surveys to more specific surveys of client satisfaction with components of individual services.
How to develop SUOs(1) Consider how your unit will determine the
extent to which it accomplishes its goals (What will you assess?)– Aim to develop SUOs that make your unit goals measurable– Use existing data as much as possible– Focus on essentials you can address– Incorporate assessment procedures into routine unit operations– Use a cycle of assessment so that all outcomes will be assessed
within a 3-year period rather than assessing all outcomes each year.
– Try to limit to 2 SUOs per functional unit
How to develop SUOs
(2) Understand the relationship between goals and outcomes
– Goals are what we plan to do. They are helpful for guiding the implementation of improvement activities but are sometimes too general, broad, or vague for developing specific tools to assess the effectiveness of services provided in support of student learning.
Goals state what we will do or cause to happen.– Outcomes are what we expect will happen when we do what we
planned. At a minimum, outcome statements should describe stakeholder behaviors that staff would accept as evidence that the support office is performing its function. Where possible and meaningful, write SUOs beginning with the client as the noun in the statement (ie., the client will receive or experience…).
Outcomes state what will happen as a result of what we do.
How to develop SUOs
Goals Outcomes
Broad, future-oriented statements Brief statements of results
What the program or office is striving toward
Describe behaviors of stakeholders
Refer to process Refer to results of processes
Used in policy making and program planning
Used to assess effectiveness and provide improvement information.
Understand the relationship between goals and SUOs, continued
Practice exercise: Goals or outcomes
• Faculty, staff, and students will have access to electronic resources from off-campus locations.
• Learning Resource Services will increase access to its resources by implementing remote user access to electronic resources.
Practice exercise: Goals or outcomes
• The Office of Institutional Effectiveness will provide planning, assessment, and evaluation services to meet the needs of the College.
• Faculty, administrators, and staff use information to make decisions or assess the effectiveness of their area.
Practice exercise: Goals or outcomes
• The Office of Risk Services will promote peace, order, and safety on campus by deterring and preventing criminal activity.
• The number of crimes reported on both the LAC and PCC campuses will decrease annually.
• Faculty, staff, and students will feel safe when they are on the campus.
Writing an Outcome Statement:The 3 M’s
1. Meaningful: How does the outcome support the departmental mission or goal?
2. Manageable: What is needed to foster the achievement of the outcome? Is the outcome realistic?
3. Measurable: How will you know if the outcome is achieved? What will be the assessment method?
- StudentVoice
See handout for more SUO examples
Why assess?
The ultimate goal of assessment is to pose questions and gather information to
bring about improvement.
Why Assess?
Student Support and Administrative units• define their mission, • establish goals • identify service unit outcomes and develop plans to assess them, and
• collect assessment dataso that key processes that meet the needs and expectations of students, parents, employers, faculty and other stakeholders can be improved on a continuous basis.
Principles of assessment
Assessment at LBCC is• To enhance quality of services provided in support of teaching and learning
• Faculty and staff driven• Conducted at the level of the unit (not about individual evaluation)
• On-going and cyclical• Done in order to improve
Assessment plan
LBCC Program/Course/Unit Level Outcome and Assessment Plan Rubric
Program/Course/Area Name: Program Mission Statement:
Intended Outcome Assessment Task Criteria /Expected Level of
Achievement Results of Assessment Use of Data/Plan
(Continue grid for each program/course/unit outcome)
Assessment plan
LBCC Program/Course/Unit Level Outcome and Assessment Plan Rubric
Program/Course/Area Name: Risk Services Program Mission Statement: …….
Intended Outcome Assessment Task Criteria /Expected Level of
Achievement Results of Assessment Use of Data/Plan
1. The number of incidents of crimes reported on campus will decline each academic year.
1. The department of Risk Services will require that the Long Beach Police Department provide an annual report on crime statistics.
Each year the number of crimes reported overall at both campuses will decline by 3%.
2. Faculty, staff and students will report feeling safe when they are on campus.
3. Items on the LBCC employee survey will query faculty and staff on the extent to which they feel safe on campus.
85% of faculty and staff responding to the LBCC employee survey will report a level of “agreement” or higher that they feel safe on campus.
(Continue grid for each program/course/unit outcome)
Note: For each functional area within a unit, do only the first 3 columns for fall 2009.
Assessment planIntended Outcome Assessment Task
Criteria /Expected Level of Achievement
Results of Assessment Use of Data/Plan
Write the intended outcome in this column.Student Learning Outcomes (SLOs) are identified in the course outline of record. Program Level Outcomes, either SLOs or service unit outcomes (SUOs) are identified statements for a program/area that will be assessed to measure the effectiveness of the program/unit.
Explain the assessment task for each outcome in this column. Make sure that a balance of direct and indirect assessment methods are used program-wide. Describe what the assessment is, how the assessment task will be implemented, who will be responsible for the task, when the assessment will be conducted, and where activities will be carried out.
The expectations for success should be explained in this column. Establish a minimum expected score for achieving the outcome (e.g. %, fraction, actual number); determine the acceptable success standard (e.g. satisfactory, a passing grade, narrative indicator); as well as the students/clients to be included (e.g. successful completers, all queried, random sample). This information initially will be based upon reflection and discussion among necessary parties. Later, consideration of previous outcomes assessment effort(s), external reports (if available) should be used for this determination.Internal and external conditions should be considered.(Conduct and document assessment activities.)
Once the assessment data is collected analyze the information through small group discussion and in relationship to previous assessment data (if available). Report the actual results and compare with the original expectations in this column. Highlight key findings from the data. Develop supportable conclusions from this information. Determinations will then be made on what action should be taken.
Report here what the department/program/unit members have done or are doing as a result of the findings. Identify any changes or steps, who has made or is making the modifications and indicate the time frame for any changes to improve the outcome. The actions should generally be substantive and detailed; although there are times when no action is necessary and this can be stated as well. Indicate what the data explains about the process and what resources are necessary to implement the needed program or course improvements to student learning.(Conduct follow-up assessment to determine the effectiveness of the program or course improvement.)
Administrative Units Completing Program Plan/
Program ReviewSuperintendent-President's
OfficeInstitutional EffectivenessCommunity Relations and
Marketing
Academic Affairs Academic Services
Administrative ServicesRisk ServicesFacilitiesFiscal ServicesInstructional and Information
Technology Services
Human ResourcesClassified AcademicProfessional Development
Student Services LBCC AuxiliaryFinancial Aid/Veteran AffairsAdmissions & RecordsCounseling/ Student Support
Development Student Affairs Outreach and Recruitment
Economic & Resource Development
Economic DevelopmentWorkforce DevelopmentInstitutional Grants
Your Unit Assignment
• Pull together a work group for your unit– Composition: Ideally, everyone (directors,
manager, supervisors, classified staff) should be involved; however, if this is not feasible, ensure that there is representation for:
• each functional area • classified staff • if applicable, for each campus and/ or for each shift
Work group assignment
• Answer two questions about your unit:– What do you do? (core functions & services)
• You may want to start by having each functional rep describe their core function. From a comprehensive list of all functions and services, distill the core functions and services of the department.
– What do you hope to improve or modify in the next 3-5 years?
• Review goals from the last Program Review. Are they still relevant?
• Draft mission statement• Draft goals for next 3-yr cycle.
Work Group Assignment
For each functional area within your unit• Draft SUOs
– Start by jotting down possible results of your goals (make your goals measurable)
• (Optional) For each of your SUOs, complete the “Assessment Task” and “Criteria for Success” columns of the Assessment Plan template
End Products from Work Group
• Draft department mission statement• Draft department goals• Draft SUOs (for each functional area
within your department)
Please have these ready and available for the customized training session.
Note: We will send you a template in which to capture the above end products from your work group.
Customized Training
• Feedback and refinement of mission, goals and SUOs
• Assessment plan• Introduction toTracDat
– Identify point person in unit to enter info into TracDat
Program/Unit description screen
Goals screen
Outcomes screen
Questions?
• Sign up for customized training