Oklahoma’s Experience: Proposed Measure Construction

for 2P1-Completion

Phoenix Data Quality InstituteJune 14-16, 2006

Kathy SpenglerOklahoma State Regents for Higher Education

Sarah MussettOklahoma Department of Career and Technology Education

Background

• Oklahoma State System of Higher Education includes 13 colleges, 2 technical branches, and 12 universities.

– Technology Centers offer certificate programs to adults at the postsecondary level.

There are two systems of public, postsecondary education in Oklahoma.

– The two-year colleges offer associate in applied science degrees, certificates of less than 1 year, and certificates between 1 and 2 years.

– Three four-year universities also offer AAS degrees.

• Career Technology System includes 29 centers

Measurement Approach

CURRENT

Denominator: Entry Cohort (1999-00) of first-time entering students who declare an AAS program as their major

Numerator:Students who are awarded a degree or certificate within six years

PROPOSED EXIT GROUP:

Denominator: 2003-04 students in an AAS or certificate program who (1) have completed 1/3 of the credits required in their program and (2) did not enroll the following year (2004-05)

Numerator:Students who were awarded a degree or certificate [or who transferred to a 2-year or 4-year institution]

Measurement Approach

1. Identify Concentrators. If the student’s educational goal (theoretically consistent with the major) = AAS or 2-year certificate level, then determine whether the cumulative credit hours >= 20. If the educational goal = 1-year certificate, then determine whether the student had earned at least 10 credit hours.

2. Identify “Exiters.” If the student was a concentrator in 2003-04, then check for enrollment in any semester of 2004-05. If no enrollment is found in that year, then flag as a “concentrator/exiter” and include in the denominator.

SYSTEMWIDE APPROACH

Measurement Approach

3. Identify Degree or Certificate Recipients. Search for degrees awarded in 2003-04 or 2004-05 to match with those students previously identified as “concentrator/exiters.” We used a two-year window to catch many of the “degree-eligible” students who may not have actually received their degrees at program completion (due to outstanding tuition/fines or late applications, etc.)

FindingsCurrent (AAS)

#

Current (AAS)

%

Exit Group (AAS only)

#

Exit Group (AAS only)

%

Exit Group (AAS &

Cert)#

Exit Group (AAS &

Cert)%

Total 7,021 27.00% 7,517 30.86% 8,110 30.74%

Male 2,978 24.11% 2,837 28.30% 3,057 28.16%

Female 4,043 29.14 % 4,680 32.41% 5,053 32.30%

Am. Indian 766 25.85 % 644 31.99% 689 32.08%

Asian 159 32.70 % 133 25.56% 145 26.21%

Black 773 15.91 % 642 21.81% 694 23.05%

Hispanic 244 23.36 % 246 24.80% 261 25.67%

White 4,933 28.83 % 5655 32.34% 6,102 31.96%

Other 146 30.14 % 197 25.38% 219 26.03%

Findings• Overall, a change to the proposed exit group method

would result in a higher Oklahoma completion rate than was reported in last year’s CAR:– Completion rates, even including certificate programs,

would be almost 4 percentage points higher.

• Comparing only AAS students (ignoring students in certificate programs) under the current and proposed completion measure constructions:– More students, in total, would be included under the

proposed method. The number of whites and females, in particular, would increase.

• Including certificates in the measure would: – improve the completion rates of minority students, up to

5.7% over what would be reported for AAS students only under the proposed plan.

Observations, Questions & Issues

• Concentrator, eligible coursework –

– “Academic and technical coursework that is part of a state established or state approved, locally established CTE program of study may count toward calculating student concentrator status.”

– Is it necessary to determine if the cumulative credit hours taken are truly “part of” the CTE program, as opposed to credits which may have been earned during previous study in a different program or may be extra electives that won’t count toward the CTE degree?

Observations, Questions & Issues

• Concentrator, program sequence – – The proposed definition states that “generally” the

average number of credit hours for an AAS degree, two-year certificate, and one-year certificate would be 60 hours, 60 hours, and 30 hours, respectively.

– Would there be an expectation that states apply program-specific requirements or institution-specific averages when determining how many hours are needed to be deemed a concentrator?

Observations, Questions & Issues

• Concentrator, student intent – – Is the use of educational goal to identify students

enrolled in AAS or certificate programs appropriate, or is it contrary to the instructions to NOT use student intent in defining a concentrator? If it is inconsistent with our proposed measure, how will other states identify students to be included?

• PROPOSED CONCENTRATOR ENTRY COHORT

– Insufficient longitudinal data were available in this trial run to identify a concentrator entry cohort.

– How difficult is it to determine the first point in time that a student attains concentrator status and avoid duplication among multiple institutions or semesters being reported?

Observations, Questions & Issues