Program Assessment Professional Development Days August 25 th and 26 th 2009.
-
Upload
jewel-kelly -
Category
Documents
-
view
213 -
download
0
Transcript of Program Assessment Professional Development Days August 25 th and 26 th 2009.
Program Assessment
Professional Development Days August 25th and 26th 2009
Workshop Learning Outcomes
• Describe the purpose of program assessment• Outline the steps in program assessment• Write measurable program learning outcomes• Choose program assessment delivery points
Your worksheets will be collected at the end and returned to you next week.
What is the Purpose?
Program “assessment is a framework for focusing faculty attention on student learning and for providing meaningful discussions of program objectives, curricular organization, pedagogy, and student development.”
Mary Allen, Assessing Academic Programs in Higher Education
The Goal is Improvement
Program assessment concentrates on the group of students in a program.
To improve:CoursesProgramsServicesPlanningBudgeting
Last but not least - Accreditation
By 2012, the ACCJC expects us to:• Have SLOs for all courses, programs and the
institution. • Have assessed SLOs at all levels• Have reported and discussed the results and
used them to guide decision-making and improvements.
Assessment Levels
• Institutional outcomes should be reflected in the programs.
• Program outcomes should be reflected in the courses.
What is a program?
It’s up to the college to decide how to group activities for assessment purposes. Some common ones:
• Sequence of courses leading to Degree or Certificate• General Education• Basic Skills• Library, Learning Assistance Center, etc.• Special sequences for groups of students like PACE,
Honors, Extension• Student services (not addressed here)
By our normal way of defining “program”, there are over 100!
How can we get it all done?
One step at a time!1.Identify assessable programs2.Develop program learning outcomes 3.Develop an assessment plan4.Collect assessment data, analyze and report5.Use results to improve the program
This workshop concentrates on 1, 2 and 3
Identify Assessable Programs
For efficiency, create groupings of programs with similar goals and courses:– CSU General Education + Graduation Plan A +
Graduation Plan B + IGETC + Liberal Studies Degree + ?
– Business + Real Estate + Accounting– Industrial Engineering + Electronic
Engineering + Engineering Technology + ?– Is Political Science a program?
STEP
1
Identify Programs
Set up a focus group to work on the next steps. It could include:
• Division chair• Faculty point person• Full time and adjunct faculty• Students
STEP
1
Worksheet and Discussion
Take a few minutes to answer the Step 1 questions on your worksheet.
STEP
1
2. Write Program SLOs
A. Review LAHC institutional SLOsB. Look at course SLOs in your programC. Search the Web for outcomes to
“borrow”D. Describe the ideal graduateE. Check draft SLOs for measurabilityF. Build course to program matrix
STEP
2
Institutional SLOs (ISLO)
5 ISLOs (handouts and campus web site).
Modify them to be specific to your program. They don’t ALL have to fit but most will.
This makes it easier for us to conduct institutional assessment!
STEP
2A
ISLO #1 Communication
Use language and non-verbal modes of expression appropriate to the audience and purpose.
Example SLO for a Child Development program:
Teach and care for young children using effective communication and relationship skills.
STEP
2A
ISLO #2 Cognition
Use critical thinking skills to analyze, synthesize and evaluate ideas and information.
Example SLO for a Math program:Estimate and check answers to
mathematical problems in order to determine reasonableness, identify alternatives, and select optimal results.
STEP
2A
ISLO #3 Information Competency
Utilize research skills necessary to achieve educational, and professional objectives.
Example SLO for a History program:Use primary and secondary sources to
construct sophisticated, persuasive, and logical interpretations of historical problems and events
STEP
2A
ISLO #4 Social Responsibility
Demonstrate sensitivity to and respect for others and participate actively in group decision making.
Example SLO for a Computer Science program:
Analyze the global impact of computing on individuals, organizations, and society.
STEP
2A
ISLO #5 Personal Development
Demonstrate self-management, maturity, and growth through practices that promote physical, mental and emotional well-being.
Example SLO for a Human Services program:Identify their personal values and articulate
how their value systems inform and influence their professional practices.
STEP
2A
Look at the course SLOs
• Find repeating– Words– Activities– Ideas
• Course assessment forms are on the campus web site (Faculty/Staff, Student Learning Outcomes/Assessment, by department)
STEP
2B
Course SLO Word Cloud
History program word cloud created at wordle.net
Search the Web • Other institutions
Use www.google.com or www.bing.com to search. Search: history program learning outcomes
Example: University of Maryland http://www.ugst.umd.edu/core/LearningOutcome.htm
• Professional Organizations– Go to the site and search for learning outcomesACM (Computer Science)ABET (Engineering)MAA (Math)Warning: you will find bad SLOs and good ones.
STEP
2C
The Ideal Graduate
• Look at program review– Mission and goals
• Advisory committee members• Transfer institution contacts– Call or email and ask what the students
transferring in need to have– What do they find lacking?
STEP
2D
The Ideal Graduate Search the Internet or ask Bob Richards for help locating data.For students studying computers:
According to Kelly IT Resources, there are a number of technical skills currently in demand, including Java 2 Platform, Enterprise Edition (J2EE), Visual Java++, WebSphere, ERP, and CRM.
However, the IT worker who wants to get ahead needs more than just technical savvy. In a survey conducted by the Journal of Computer Information Systems, results indicated that attributes including, “interpersonal and personal skills/traits, such as team skills, communication skills, critical/creative thinking skills, and personal motivation” are often valued higher than the traditional technical requirements of a job.
From “IN-DEMAND IT SKILL SETS: WHAT EMPLOYERS WANT, WHAT WORKERS NEED” Published by Kelly IT Resources at http://www.kellyit.com.pr/res/content/us/kit/en/docs/kitr_in-demand_skillsets.pdf
STEP
2D
Check: Measurability
• Use active and specific terms. See Bloom’s taxonomy (in handouts)
• Don’t use words like “understand” – go for higher level skills
• The meaning of a word can be defined more clearly in the scoring rubric. For example, “sophisticated” from the history SLO.
STEP
2E
Check: Simplicity
• Keep the number down to around 5• Have more if your outcomes are
determined by an outside accrediting body like Nursing.
• Concentrate on the most important skills or knowledge the students will come away from the program with and what they will produce to show you that they have mastered those skills.
STEP
2E
Worksheet and Discussion
Take a few minutes to answer the Step 2 questions on your worksheet.
STEP
2E
Check Alignment with CoursesCurriculum Alignment Matrix
Course PSLO#1
PSLO#2
PSLO#3
PSLO#4
PSLO#5
PSLO #6
100 I I
120 I P
200 P P P
204 P
300 P P
329 D P
400 P D
480
490 D D D D
A matrix ensures that student have been exposed to the outcome in the program.
Can point out problems!
From Assessing Academic Programs in Higher Education by Mary J. Allen p. 43
STEP
2F
Program Assessment Form
– Blank form is a document on the campus Assessment Web site
– Download and type in your program SLOs with Word
– Send document to Division Chair who will forward to Dean and Assessment Coordinator for Web publication.
– Program Learning Outcomes are also to be published in the college catalog and all other program materials.
STEP
2
Develop Assessment Plan
A. Select delivery points for assessmentB. Select tool(s)C. Set timelineD. Create assessment and rubric
STEP
3
Delivery Point
Is there a course or set of courses all majors would take before graduating?
If not, is there a course many majors take? Give a survey to identify who is a major.
See Accounting program example on handouts.
STEP
3A
Delivery Point
If there isn’t a terminal course,• Can you reach students as they are
graduating to give out an assessment?– One campus had a free lunch for students
who filed petitions
• Can you have students build a portfolio as they progress through the program?
STEP
3A
Worksheet and Discussion
Take a few minutes to answer the Step 3 questions on your worksheet.
STEP
3A
Choose Assessment Tool(s)
• Direct Assessment• Indirect AssessmentExperts recommend more than one type of
data collection method!
STEP
3B
Choose Assessment Tool(s)Direct methods prompt students to represent of
demonstrate their learning or produce work so that observers can assess how well students’ texts or responses fit expectations
Includes:Standardized testsLocally developed tests and assignmentsProjectsPresentationsPortfolios
See handouts
STEP
3B
Choose Assessment Tool(s)
Indirect methods capture students’ perceptions of their learning and the educational environment that supports that learning.– Student surveys– Factbook information
Definitions of direct and indirect from Assessing for Learning by Peggy Maki p. 88
STEP
3B
Set Timeline
– Which semester/year will each SLO be assessed during?
– The general recommendation is to measure one per year to give time for careful review of results and integration into planning
– Send the timeline to the Division Chair and to the Assessment Coordinator
STEP
3C
Create Assessment and Rubric
– Develop the tool (test questions, assignment, survey, etc.)
– Create the scoring guide or rubric– Get input from all of the faculty who will be
involved.
STEP
3D
For Future Workshops?
• Designing assessments and rubrics• Collecting assessment data, analyzing
and reporting• Using results to improve the program
You tell us!
Thank You For Attending!
• Please turn in:The WorksheetThe Assessment Needs SurveyThe evaluation form
• Presentation will be available on the campus assessment web pages.
• Questions? – Lora Lane x4178 or campus email– Dean Humphreys x4025 or campus email