Post on 05-Jan-2016
description
The work reported herein was supported under the National Research Center for Career and Technical Education, PR/Award (No. VO51A070003) as administered by the Office of Vocational and Adult Education, U.S. Department of Education. However, the contents do not necessarily represent the positions or policies of the Office of Vocational and Adult Education or the U.S. Department of Education and you should not assume endorsement by the Federal Government.
Pradeep Kotamraju, Ph.D.Deputy Director
Principal Investigator
Oscar Aliaga Ph.D.Program Director
Emily DickinsonGraduate Assistant
Year 2
Year 3
Year 4
Technical Skills
Crosswalk Validation
NCES Data Analysis
ROI
DevelopedInventory Template
PopulatedInventory Template with Data Submitted by States
Linked 3rd Party
Industry Certification to LMI
Figure 1:The Work So Far
Built a Common Postsecondary
CTE Data Dictionary
Analyzed Data To Test Common
Postsecondary CTE Data Dictionary
Jointly with the Center for
Education and the Workforce
(CEW), Georgetown U,
Producing a Report on 2018
Employment Projections by
Career Clusters
Working on NCES Restricted Sample
Survey Data to Analyze CTE Engagement,
Achievement, and Transitions
Community College
Completions Using IPEDS
Data; Published in Community
College Journal of Research &
Practice
Developing an ROI Guidebook
Neutral Intermediary
Chaired NSWG on POS
Measurement, Accountability, and Evaluation
Produced a Brief on ROI for the
USA TVET Consortium
Made a Presentation on ROI at the ACTE
Annual Conference
An NRCCTE Report: Common Postsecondary
CTE Data Dictionary
Technical Skills
Crosswalk Validation
NCES Data Analysis
ROI
Produce a Final
Inventory
Conduct Two Sub-Analyses (one on Soft
Skills and the Other on Technical Skills and Connect to
CTE Engagement, Achievement,
and Transitions Using State-level Data
Figure 2:The Work For Year 5 (and Beyond?)
Develop and Validate Existing
OVAE/Career Clusters Crosswalk
State-Level Analysis on
2018 Employment Projections by Career Clusters/ Pathways
Continue Work on NCES Restricted Sample Survey Data to Analyze
CTE Engagement, Achievement, and
Transitions
Refining the ROI Guidebook
Neutral Intermediary
Produce Presentations,
Reports, Monographs,
Journal Article Submissions
Preliminary Analysis of
Using Existing Data Sets to
Develop ROI for CTE
As requested by Center Partners and by External
Stakeholders
In Center Years 2 and 3 (August 1, 2008-July 31, 2010), data collection began and continued in which states were asked to submit information that corresponded with the elements of the template. Fourteen states submitted information that has been reformatted and entered into the template. Analysis of the information points to the following: The top seven program areas (based on two-digit CIP codes) are IT,
Health, Business, Manufacturing, Construction, Communications, and Agriculture.
Third-Party Certification/Technical Exams, State-Developed Program of Study Exams, and State-Developed End-of-Program Exams appear to be most commonly used in states.
Outstanding issues to be resolved include: Status of Postsecondary TSAs (no information has been provided as
yet to the Center). Program of Study level assessments – the extent to which the
postsecondary partner has a role and input. The shape, form and extent of the relationship of states with third-
party TSA organizations.
4
Technical Skills Assessment
In Center Year 4(August 1, 2010-July 31, 2011), the NRCCTE was requested by the Office of Vocational and Adult Education (OVAE), U.S. Department of Education (USDE) to connect industry skill certifications, currently being offered within different state educational programs, to available labor market information (LMI).
The analysis required obtaining data on industry skill certifications, educational program and CIP codes, labor market information, and a crosswalk connecting CIP codes and occupational (SOC) data. The methodology used:
Converted the data states submitted to NRCCTE as part of the technical skills assessment (TSA) inventory template project into a database that could be used in the analysis.
The template data was collected in a Microsoft Excel spreadsheet. The spreadsheet information was reformatted so that it could be imported into a Microsoft Access database.
5
Technical Skills Assessment
The crosswalk information, which was only available as a Microsoft Excel spreadsheet, was also reformatted for import into the same Microsoft Access database.
The wage information was downloaded as a Microsoft Excel workbook from the U.S. Department of Labor and appropriate wage information was reformatted for import into the same Microsoft Access database. A database with all connected information was then assembled.
The database contains 139 third-party, industry-based technical skill certifications, 111 secondary education programs, 56 CIP codes, and 110 SOC codes for the five states.
6
Technical Skills Assessment
7
CIP Family
Arts, Design, Entertainment,
Sports and Media
Construction and Extraction
Healthcare Practioner and
Technical
Healthcare Support
Installation, Maintenance,
and Repair
Office and Administrative
Support
Personal Care and Service
Production Protective Services
Sales and Related
Transportation and Material
Moving
Total Industry Skill
Certification
AGRICULTURE, AGRICULTURE OPERATIONS, AND RELATED
SCIENCES.1 1 2 1 5
BUSINESS, MANAGEMENT, MARKETING, AND RELATED
SUPPORT SERVICES. 4 2 2 8
COMMUNICATIONS TECHNOLOGIES/
TECHNICIANS AND SUPPORT SERVICES.
1 2 10 13
COMPUTER AND INFORMATION SCIENCES AND
SUPPORT SERVICES. 22 22
CONSTRUCTION TRADES. 112 7 18 8 145ENGINEERING TECHNOLOGIES AND ENGINEERING-RELATED
FIELDS.1 1
FAMILY AND CONSUMER SCIENCES/HUMAN SCIENCES. 3 3
HEALTH PROFESSIONS AND RELATED PROGRAMS. 1 6 2 9
HOMELAND SECURITY, LAW ENFORCEMENT,
FIREFIGHTING AND RELATED PROTECTIVE SERVICES.
4 4
MECHANIC AND REPAIR TECHNOLOGIES/
TECHNICIANS. 5 31 24 1 61
PERSONAL AND CULINARY SERVICES. 5 2 7
PRECISION PRODUCTION. 22 22TRANSPORTATION AND MATERIALS MOVING. 10 1 2 13
VISUAL AND PERFORMING ARTS. 4 4
All CIP Family 7 127 1 7 38 33 5 81 4 3 11 317
Tables 1: Relating CIP and Occupation Families: No. of 3rd Party Industry Certifications
8
CIP FamilyArts, Design,
Entertainment, Sports and Media
Construction and Extraction
Healthcare Practioner and
Technical
Healthcare Support
Installation, Maintenance,
and Repair
Office and Administrative
Support
Personal Care and Service
Production Protective Services
Sales and Related
Transportation and Material
MovingTotal Wage
AGRICULTURE, AGRICULTURE
OPERATIONS, AND RELATED SCIENCES. $19,630 $17,190 $16,010 $15,680 $17,128
BUSINESS, MANAGEMENT, MARKETING, AND RELATED
SUPPORT SERVICES.$24,835 $22,920 $15,730 $21,162
COMMUNICATIONS TECHNOLOGIES/
TECHNICIANS AND SUPPORT SERVICES. $20,210 $18,195 $18,860 $19,088
COMPUTER AND INFORMATION SCIENCES AND SUPPORT SERVICES. $23,925 $23,925CONSTRUCTION TRADES. $25,453 $29,249 $19,390 $29,448 $25,885
ENGINEERING TECHNOLOGIES AND
ENGINEERING-RELATED FIELDS. $24,620 $24,620
FAMILY AND CONSUMER SCIENCES/HUMAN
SCIENCES. $16,230 $16,230HEALTH PROFESSIONS AND
RELATED PROGRAMS. $21,610 $19,322 $20,410 $20,447HOMELAND SECURITY, LAW ENFORCEMENT, FIREFIGHTING AND
RELATED PROTECTIVE SERVICES. $23,990 $23,990
MECHANIC AND REPAIR TECHNOLOGIES/
TECHNICIANS. $28,922 $23,005 $22,344 $27,220 $25,373PERSONAL AND CULINARY
SERVICES. $23,298 $16,890 $20,094
PRECISION PRODUCTION. $21,820 $21,820TRANSPORTATION AND MATERIALS MOVING. $26,104 $29,500 $18,855 $24,820
VISUAL AND PERFORMING ARTS. $24,620 $24,620
All CIP Family $22,270 $26,826 $21,610 $18,256 $26,127 $22,563 $16,120 $21,517 $23,990 $15,705 $25,174 $21,833
Table 2: Relating CIP and Occupation Families: Average Entry-Wage for 3rd Party Industry Certifications
The NRCCTE convened a workgroup consisting of researchers, data experts, and policymakers who met on January 12th and 13th, 2010, at the University of Louisville. The purpose of this meeting was to capture the evolving multi-state, multi-institutional collaborative efforts seeking greater consistency and clarity in Perkins secondary and postsecondary data collection and reporting, The goal of this meeting was to continue to move toward a common data crosswalk that links educational programs to occupations with career clusters and career pathways and create a foundation for more standardized accountability requirements in later iterations of the Perkins legislation. Concluding the meeting, participants recommended that the Center undertake the following tasks: Document and validate the decision rules that connect occupations to educational
programs, career clusters, and career pathways in several of the existing crosswalks.
Analyze and discuss the different units of analyses that are currently operational within several crosswalks, and outline for a plan for incorporating programs of study as a unit of analysis for crosswalks.
Develop a plan for disseminating the procedures, processes and products to data experts, researchers and policymakers.
The project continues into Year 4 and is being jointly led by the University
of Louisville and the NASDCTEc. Jointly with the Center for Education and the Workforce (CEW),
Georgetown U, Producing a Report on 2018 Employment Projections by Career Clusters
9
Crosswalk Validation Project
Traditionally, only the total number of CTE credits enrolled and completed either across all areas or within one area are used to identify the extent of participating or concentrating in CTE. A three-credit threshold is used to divide CTE students into participants and concentrators
The NRCCTE has begun exploring, examining, and analyzing credits taken within an occupational area, across all occupational areas, and the number of occupational areas in which high school graduates participate and concentrate
The main source is the US Department of Education sample survey data, specifically the 2005 NCES High School Transcript Studies (HSTS) data and the Education Longitudinal Study (ELS) data
10
The NRCCTE CTE TYPOLOGY
11
Conventional Way of Defining A CTE student
Different denominations: concentrator, participant, investor …
“Traditional” classification in large surveys:Academic trackVocational trackNeither and both (dual)—as defined in the NCES
sample surveyNCES: Fulfillment of an occupational area if 3 or
more credits are taken in that area (“occupational concentrator”)
At the State Level: different levels, different goals
A Methodological Rationale for
Measuring, Accounting, and Evaluating CTE
The purpose is to understand, the level, mix, and intensity of CTE course-taking within and across different occupational areas.
The NRCCTE has expanded the participant-concentrator dichotomy by creating a taxonomy that more accurately represents CTE course-taking.
Using the Classification of Secondary School Courses (CSSC) different parts of the high school graduates’ transcripts are analyzed
Reformulating CTE Course Taking
14
• Following along the lines of NCES, the basic steps are as follows:
Identify CTE and Academic courses Divided CTE into 13 occupational areas Divided Academic into different subject areas,
including Math and Science Categorized Math & Science courses by three
levels: Less than Basic, Basic, and Advanced Identified Academic and CTE courses which are
in sequence and further divided by No Sequence/Sequence
Determined the last grade year Math & Science course taken
Reformulating CTE Course Taking
Reformulating CTE Course Taking
Dividing CTE course taking patterns of high school graduates (students) into three facets: Identify how many credits high school graduates
takes and completes within each occupational area
Determine the total number of CTE credits enrolled and completed across all occupational areas
Count the number of occupational areas in which each graduate enrolls and completes courses
A crosstab has been built that relates the above three facets creating the following (shown on the next slide)
16
Reformulating CTE Course Taking
No CTE Course Taking
Concentrator
Participant
0 is No CTE Course Taking; 1,2, and 3 are classified as Participants; 4 is classified as Concentrators
17
Reformulating CTE Course Taking
No CTE Course Taking
Concentrator
Participant
0 is No CTE Course Taking; 1,2, and 3 are classified as Participants; 4 is classified as Concentrators
18
CTE Credit Categorization %
No CTE credits 8.0
More than 0 and less than 1 CTE credits, no occupational area fulfilled7.5
1 CTE credit, no occupational area fulfilled 10.7
More than 1 and less than 3 CTE credits, no occupational area fulfilled29.9
3 CTE credits, no occupational area fulfilled 7.6
More than 3 CTE credits, no occupational area fulfilled 19.6
3 CTE credits, 1 occupational area fulfilled 0.8
More than 3 credits, at least 1 occupational area fulfilled 16.0
Frequencies of CTE Credit Categorizations (n=2,698,609)
19
0 1 2 3 4 5 6 7 Weighted n
Female 9.0 8.1 11.4 30.8 7.6 18.4 0.9 13.8 1,378,033Male 6.9 6.8 9.9 28.9 7.6 20.8 0.7 18.3 1,320,577
White 8.3 7.4 10.1 27.8 7.1 21.2 0.8 17.4 1,633,287Black 8.0 5.7 8.9 33.5 7.8 17.9 1.3 16.9 379,133Hispanic 7.7 9.2 12.2 31.6 8.8 17.7 0.8 12.1 421,325Asian 10.3 9.8 17.0 34.9 7.2 12.0 0.3 8.4 122,181Other 7.5 9.8 21.0 70.2 16.5 46.8 1.3 26.7 142,684
SES-Q1 (Low) 5.9 6.0 9.7 29.5 7.8 20.6 0.9 19.6 640,172SES-Q2 5.7 6.6 8.6 28.9 7.2 23.6 0.5 19.0 687,844SES-Q3 8.7 6.4 10.3 30.7 8.2 19.5 0.8 15.3 703,344SES-Q4 (High) 11.7 11.0 14.1 30.4 7.1 14.5 1.0 10.2 667,249
Group 0: No CTE creditsGroup 1: More than 0 and less than 1 CTE credits, no occupational area fulfilledGroup 2: 1 CTE credit, no occupational area fulfilledGroup 3: More than 1 and less than 3 CTE credits, no occupational area fulfilledGroup 4: 3 CTE credits, no occupational area fulfilledGroup 5: More than 3 CTE credits, no occupational area fulfilledGroup 6: 3 CTE credits, 1 occupational area fulfilledGroup 7: More than 3 credits, at least 1 occupational area fulfilled
Frequencies of Gender, Race and Socioeconomic Status by CTE Credit Categorizations for Public High School Students
Students taking more than 3 CTE credits, with or without a focus on an occupational area, are 36% to 59% less likely to drop out of high school than those taking fewer CTE credits.
Students who have taken 3 or more CTE credits, fulfilling the requirements of an occupational area or not, are at least 43% more likely to attend 2-year colleges than those taking fewer CTE credits.
Based on high school graduate transcripts that describe CTE and academic course taking, and using conventional definitions of college, career, and college and career readiness, early estimates for each are 42%, 28%, and 18%, respectively.
20
Preliminary Results
21
Kotamraju, P. (2011, June). A new taxonomy of high school career and technical education (CTE): Describing the engagement, achievement, and transitions of CTE students. Paper presented at the 15h Annual Career Clusters Institute, Atlanta, GA.
Kotamraju, P. (in press—expected publication date, Summer 2011). Vocational education and training quality and evaluation: Its place in the U.S. community college. In A. Barabasch & F. Rauner (Eds.), The art of integration: Work and education in America. Berlin, Germany: Springer Verlag Press.
Kotamraju, P. (2011). Meeting the 2020 American Graduation Initiative (AGI) goal of increasing postsecondary graduation rates and completions: A macro perspective of community college student educational attainment. Community College Journal of Research and Practice, 35(1-2), 1-18.
Kotamraju P., Richards, A., Wu, J., & Klein, S. (2010, April). A common postsecondary data dictionary for Perkins accountability. Louisville, KY, and Berkeley, CA: National Research Center for Career and Technical Education, University of Louisville, and MPR Associates, Inc. Retrieved from http://136.165.122.102/UserFiles/File/Tech_Reports/Postsecondary_Data_Dictionary_Report_WEB.pdf
22
Stakeholder Contacts: Publications
Kotamraju, P. (2011, May). CTE Measurement, Accountability and Evaluation: The NRCCTE’s Comprehensive Strategy for Technical Assistance. Paper To Be Presented at the National Association of Career and Technical Education Information annual Meeting, Philadelphia, Pennsylvania.
Kotamraju, P. (2011, May). A New Taxonomy for Career and Technical Education (CTE): Measuring Engagement, Achievement, and Transitions of CTE Students. Paper To Be Presented at the National Association of Career and Technical Education Information annual Meeting, Philadelphia, Pennsylvania.
Kotamraju, P. (2011, April). Getting Timing and Sequencing of Math and Science Course-Taking Right: A Possible Answer to the College and Career Readiness of High School Graduates Concentrating on Career and Technical Education (CTE). Paper presented at 2011 Council for the Study of Community Colleges Annual Conference. New Orleans.
Aliaga, Oscar. (2011, April). Career and Technical Education Graduates and Enrollment in 2-Year Colleges. Paper presented at the annual meeting of the Council for the Study of Community Colleges. New Orleans, Louisiana.
Kotamraju, P. (2011, April). The college and career readiness of U.S. high school graduates: From concept to measurement. Paper to be presented at the annual convention of the American Educational Research Association. New Orleans, LA.
23
Stakeholder Contacts: Paper Presentations
Kotamraju, P. (2010, December). A tool kit for measuring CTE effectiveness using return on investment and other related techniques: A basic introduction. Paper presented at the annual convention of the Association for Career and Technical Education, Las Vegas, NV.
Kotamraju, P. (2010, October). Meeting the 2020 AGI completion goal: The role of the community college. Paper presented at the annual convention of the National Council of Workforce Education, Washington, DC.
Kotamraju, P. (2010, June). Building a Technical Skills Inventory Database One State at a Time,” 14th Annual Career Clusters Institute, Denver, Colorado.
Kotamraju, P. (2010, May). A Cross-State Comparison of Postsecondary CTE Student Graduation Rates and Completions: Determining the Efficacy of Using IPEDS Data for Perkins Reporting,” National Association of Career and Technical Education Information Annual Meeting, St. Louis, Missouri.
Kotamraju, P. (2010, May). Building a Useable Inventory Template for Collecting State and Local Information for Meeting the Technical Skills Accountability. National Association of Career and Technical Education Information Annual Meeting, St. Louis, Missouri.
Kotamraju, P. (2010, April). A Cross-State Comparison of Postsecondary Student Graduation Rates: A Macro Perspective of Educational Attainment,” Annual Meeting, Council for the Study of Community Colleges, Seattle, Washington.
24
Stakeholder Contacts: Paper Presentations
Kotamraju, P. (2009, December). The Minnesota FastTRAC Project: Solving the Career Pathways Puzzle. 15th Annual Data Quality Institute, Baltimore, Maryland.
Kotamraju, P. (2009, December). Building a Technical Skill Inventory Database One State at a Time,” 15th Annual Data Quality Institute, Baltimore.
Kotamraju, P. (2009, May). A Repository Template of Technical Skills Assessment: Collecting, Accessing, Reporting and Using Information and Resources for Meeting the Perkins Technical Skill Attainment Indicator,” National Association of Career and Technical Education Information Albuquerque, New Mexico.
Kotamraju, P. (2009, May). Building a Data Dictionary Template from State Administrative Record Data Systems,” Albuquerque, New Mexico.
25
Stakeholder Contacts: Paper Presentations
26
Correspondence should be sent to:
Pradeep Kotamraju, NRCCTE, College of Education and Human Development
Room 351 University of Louisville, Louisville, KY 40292.
Phone: 502-852-7714. Email: pradeep.kotamraju@nrccte.org.