Specific outcomes can be compared to resources expended; and successful programs can be highlighted;

Programs that have not accomplished their intended outcomes can be identified;

The extent to which the planned outcomes were met can be documented;

The adequacy of the program’s outcomes as related to the problem that the program was developed and implemented to address can be documented;

Strengths and weaknesses of each activity can be identified that affected the Effectiveness and Adequacy of the programmatic inputs ($$$, human resources, space, etc.) in meeting the activity’s planned outcomes.

The institution has the ability to determine the impact of the Title III funds awarded to various discrete activities and programs.

• The process determines not only the success or failure of the program, but also provides explanations for the actual outcomes.

As a result of the MANDATE of funding agencies and institutions to demonstrate programmatic accountability, one of the services requested most by institutions of higher education is program ASSESSMENT/EVALUATION.

•NCCU, chartered in 1909, opened its doors to students in 1910 as the National Religious Training School and Chautauqua.

• North Carolina Central University is a part of the University of North Carolina System, made up of 16 individual campuses (17 when we count the School of the Arts) and headed by a single president and governed by the University of North Carolina Board of Governors.

 

• North Carolina Central University is a comprehensive university offering bachelor’s degrees in more than 100 disciplines, master’s degrees in more than 40 programs and three professional degrees in law.

• Enrollment at North Carolina Central University increased from 5,753 in Fall 2001 to 8, 619 for Fall 2006, an increase over the six year period of 33%. We currently enroll 8,500 students (2010-2011).

• Major emphasis for the current planning

period (2007-2012) is “Enhancing Retention, Graduation, and Placement Rates through Increased Student Learning.”

Our goals: • to provide programs, activities, and services that

will assist students in becoming successful during their tenure at the University so that they persist and graduate and are successful in the career markets of the future;

• to put in place those systems designed to make the matriculation more seamless and easier to negotiate;

• to provide the training and upgrading of the skills of our faculty to work with those students whom we enroll that have been underserved and are from diverse backgrounds; and

• to promote a more efficient, effective, and accountable environment as relates to student learning and student success.

• Impetus for conducting an Impact Study – Chancellor Charlie Nelms;

• A directive he gave me as Title III Director, which I passed on to the Activity Coordinators in our planning meeting;

• His absolute need to have concrete documentation of the impact that our Title III dollars had had over the years.

North Carolina Central University (NCCU) contracted with Associates for Institutional Development, Inc. (AID, Inc.) to conduct an Impact Study of select Title III activities encompassing the grant years 2004-2009 to document the extent to which each activity had accomplished the outcomes for which the activity was funded.

•PROVIDE ACCESS TO ALL RECORDS

PERTINENT TO EACH ACTIVITY

WORK CLOSELY WITH THE TITLE

III STAFF TO COLLECT THE

DATA

A data collection process MUST be implemented that includes:

1.Onsite visits; 2.Interviews with key staff; 3.Participant self-assessments;

4. Interviews with faculty, students, and external stakeholders;

5. The administration of surveys and questionnaires, as needed.

To guarantee objectivity, the process relies upon multiple sources of data encompassing multiple points of view. Finally, to be adaptable, the assessment’s design must meet the assessment needs of the institution.

• PROGRAM PLANS (RATIONALE, TARGET AUDIENCE, BASELINE DATA;

• MEASURABLE OBJECTIVES;• MEASURABLE AND TIME-SPECIFIC

PERFORMANCE INDICATORS; • IMPLEMENTATION STRATEGIES IN

CHRONOLOGICAL ORDER

 

• DOCUMENTATION ON ACTUAL PROGRAMMATIC OUTCOMES (DESCRIPTIVE AND STATISTICAL)

• EXTERNAL EVALUATION REPORTS

• PROGRESS REPORTS (MONTHLY OR QUARTERLY)

• PROGRAMMATIC OUTCOMES

(QUANTIFIABLE ANDDESCRIPTIVE)

• ANNUAL REPORTS • RESULTS OF SURVEYS AND

QUESTIONNAIRES• FUNDS AWARDED EACH

ACTIVITY DURING THE YEARS INCLUDED IN THE STUDY

• ACTIVITY BUDGET BURN RATES

• ANEDOTAL INFORMATION THAT WILL ADD VALUE TO THE OUTCOMES

• OBJECTIVES THAT ARE NOT MEASURABLE• TOO MANY TASKS MASQUERADING

AS OBJECTIVES• PERFORMANCE INDICATORS THAT

ARE NOT MEASURABLE AND NOT TIME-SPECIFIC

• THE ABSENCE OF BASELINE DATA FOR THE FIRST YEAR OF IMPLEMENTATION AND EACH YEAR OF FUNDING (WHERE APPROPRIATE)

• LACK OF ANNUAL EVALUATION REPORTS OR SOME OTHER TYPE OF ASSESSMENT TO VERIFY THE ATTAINMENT OF THE OBJECTIVES EACH YEAR OF FUNDING

• ACTIVITY DIRECTORS NOT HAVING A CLEAR VISION OF THE PRIMARY ANTICIPATED/PLANNED OUTCOMES OF THE ACTIVITY AND HOW THE ACTIVITY WILL STRENGTHEN THE INSTITUTION

• Identified  the  over-arching goal  or  planned outcome  for each program;• Identified  the actual outcome based on data; 

• Analyzed the gap between the desired  outcome  and  the actual  outcome  by  utilizing the  “Effectiveness Estimate (EE)” technique:

• DOCUMENTS: THE EFFECTIVENESS OF THE

PROGRAM IN AMELIORATING OR STRENGTHENING THE ENTITY FOR WHICH IT WAS FUNDED

• (R) Actual Results

• (C) Results Without Program

• (P) Planned Impact

• (R) Actual Results

• (C) Results Without Program

ADEQUACY RATIO = (R) Actual Results

1 - (C) Results Without Program

1. Institutional enhancements as a result of implementing the program;

2. Enhancements to student and faculty performance, where applicable;

3. Affect on administrative personnel effectiveness, where applicable;

4. Overall academic and/or administrative program effectiveness enhancements;

5. Problems encountered that adversely

affected the implementation of the program during the specified time- period;

• Recommendations to strengthen the continued implementation of each program regardless of funding status;

Information garnered through the impact study process, in combination with other sources of data, will provide an opportunity for the institutional leadership to incorporate the findings into programmatic and budgetary decision-making.

• Dr. Gloria P. James: gpryorjames@comcast.net

• Dr. Brenda R. Shaw:  bshaw@nccu.edu

• Mr. Samuel T. Rhoades:  strhoades@aol.com