Reconsidering the Scientists and Engineers Statistical ...SESTAT • The NSF’s Scientists and...

Reconsidering the Scientists and Engineers

Statistical Data System

(SESTAT)

John Finamore

CGS Graduate Education Research & Policy Forum

January 18, 2012

National Science Foundation

National Center for Science and Engineering Statistics

www.nsf.gov/statistics/

Overview

• What is SESTAT?

• The Development of SESTAT

• Reconsidering SESTAT

• Data Needs and Analytical Interests

1

What is SESTAT?

SESTAT

• The NSF’s Scientists and Engineers Statistical Data System

(SESTAT) is a unique source of cross-sectional and

longitudinal information on the college-educated U.S.

science and engineering workforce

• The SESTAT data are collected through three biennial

surveys:

- National Survey of College Graduates (NSCG)

- National Survey of Recent College Graduates (NSRCG)

- Survey of Doctorate Recipients (SDR)

3

SESTAT Survey Questionnaires

• The core questionnaire items are similar across the three

SESTAT surveys

• All surveys have similar questionnaire sections:

- Education background

- Employment situation (Current and Past)

- Other work-related experiences

- Demographic information

• SESTAT provides information on topics such as

educational attainment, employment status, work

activities, salary, and continuing educational experiences 4

SESTAT Questionnaire Items

• Question modules rotate in and out of the surveys

throughout the decade

• For the 2010 survey cycle, new questionnaire items

include:

- Importance of various job attributes

- Available benefits in principal job

• For the 2010 survey cycle, past items rotated in

included:

- Satisfaction with various job attributes

- Membership in professional meetings

- Immigration module 5

The Development of SESTAT

NSF’s Scientists and Engineers Data

• For over 60 years, the National Science Foundation (NSF)

has been responsible for providing information about the

nation’s scientists and engineers

• In response to a 1989 request to review the NSF’s system

on scientists and engineers data, the Committee on National

Statistics (CNSTAT) made the following recommendations

7

1989 CNSTAT Recommendations

Recommendation 5.6.

NSF should conduct a large postcensal survey based on the

decennial census long form that provides baseline

information on college graduates

Recommendation 5.7.

NSF should conduct a panel survey that includes a sample

for each new graduating class of students at higher

educational institutions

Recommendation 5.8.

NSF should continue the Survey of Doctorate Recipients as

a major source of information on science and engineering

personnel at the Ph.D. level 8

Implementation of Recommendations

NSF implemented the 1989 CNSTAT recommendations

through the coordinated sponsorship of three surveys:

• National Survey of College Graduates (NSCG)

• National Survey of Recent College Graduates (NSRCG)

• Survey of Doctorate Recipients (SDR)

The information from these three surveys were combined to

form a data system called the Scientists and Engineers

Statistical Data System (SESTAT)

9

SESTAT Target Population

Each SESTAT survey was designed to target a unique

population

• The NSCG: all college graduates residing in the

U.S. as of the decennial census reference date

• The NSRCG: recent college graduates with a

bachelor’s or master’s S&E degree earned from a

U.S educational institution

• The SDR: all U.S.-earned S&E doctoral degree

recipients including those earned during the

previous two academic years 10

Type of Degree Holder Date Degree Earned SESTAT Survey

Foreign-Earned S&E

Degree Holders

Prior to April 2000 NSCG

After April 2000

U.S. Earned S&E

Bachelor's and Master's

Degree Holders


2001-2002

2003-2005

2006-2007

2008-2009

U.S. Earned S&E Doctoral

Degree Holders


2001-2002

2003-2005

2006-2007

2008-2009

11

Note: SESTAT also provides coverage of the population of U.S. residents with a non-S&E degree

working in an S&E occupation. In the 2010 decade, SESTAT will continue to cover this population,

and will also provide coverage of the non-S&E degree holders working in a non-S&E occupation.

SESTAT Coverage During the 2000 Decade


Foreign-Earned S&E

Degree Holders


After April 2000

U.S. Earned S&E


Degree Holders


2001-2002 RCG (2003)

2003-2005

2006-2007

2008-2009


Degree Holders


2001-2002

2003-2005

2006-2007

2008-2009

12



Foreign-Earned S&E

Degree Holders


After April 2000

U.S. Earned S&E


Degree Holders


2001-2002 RCG (2003)

2003-2005 RCG (2006)

2006-2007 RCG (2008)

2008-2009 RCG (2010)


Degree Holders


2001-2002

2003-2005

2006-2007

2008-2009

13



Foreign-Earned S&E

Degree Holders


After April 2000

U.S. Earned S&E


Degree Holders


2001-2002 RCG (2003)

2003-2005 RCG (2006)

2006-2007 RCG (2008)

2008-2009 RCG (2010)


Degree Holders

Prior to April 2000 NSCG & SDR

2001-2002 SDR (2003)

2003-2005

2006-2007

2008-2009

14




Foreign-Earned S&E

Degree Holders


After April 2000

U.S. Earned S&E


Degree Holders


2001-2002 RCG (2003)

2003-2005 RCG (2006)

2006-2007 RCG (2008)

2008-2009 RCG (2010)


Degree Holders


2001-2002 SDR (2003)

2003-2005 SDR (2006)

2006-2007 SDR (2008)

2008-2009 SDR (2010)

15

The Importance of the NSRCG

• In the 1990 and 2000 decades, the NSCG did not

capture the inflow of two groups into the S&E workforce:

- Recent college graduates over the decade

- Immigrants into the U.S. over the decade

• The goal of NSRCG is to improve the coverage of

NSCG. Without the NSRCG, SESTAT would not provide

data for recent college graduates with a bachelor’s or

master’s S&E degree

16

NSRCG

• Cross-sectional survey of individuals who recently received

bachelor’s or master’s degrees in S&E fields from U.S.

academic institutions – the inflow of new graduates through

the decade

• Collected using a two-stage sample design

Stage 1 – U.S. educational institutions

Stage 2 – Graduates within sampled institutions

• Most expensive of the SESTAT surveys

- two-stage sample design (institutions and graduates)

- hard to reach, mobile population 17

Reconsidering SESTAT

Examining the Potential of the ACS

• The long form was the basis of the “once per decade”

sample for the NSCG in the 1990 and 2000 decades

• After the 2000 decennial census, the Census Bureau

discontinued the use of the long form and introduced the

American Community Survey (ACS)

• The ACS is an on-going survey that provides national,

state, and metropolitan level estimates of the

characteristics of the U.S. population

19

2008 CNSTAT Recommendation

• In light of this change from the long form to the ACS as

the NSCG sampling frame, CNSTAT was charged with

assessing the benefits of the ACS for the NSF

• In response, CNSTAT provided the following

recommendation

Recommendation 7.5.

The NSF should use the opportunity afforded by the

introduction of the ACS as a sampling frame to reconsider

the design of the SESTAT Program and the content of its

component surveys

20

Motivation for Reconsidering SESTAT

• Coverage

- The American Community Survey (ACS) is being

used to draw sample for the NSCG on a biennial

basis

- The ACS-based rotating panel design of the

NSCG now provides coverage of the inflow of

recent college graduates

21

Motivation for Reconsidering SESTAT

(Continued)

• Timeliness

- Over the years increasing complexity has been

built into the SESTAT design

- There is a need to improve the timeliness of the

data releases

• Cost

- The reality of budgetary pressures now and into

the future

- The rapidly increasing costs of surveys

22

SESTAT Redesign Possibility

• One possibility is to redesign SESTAT to include only

the SDR and NSCG

• In this design, we would include more young college

graduates into our NSCG sample as a replacement for

the recent college graduates sample from the previous

SESTAT design

23


Current Design

24

Note: SESTAT also provides coverage of the population of U.S. residents with a non-S&E degree

working in an S&E occupation. In the 2010 decade, SESTAT will continue to cover this population,

and will also provide coverage of the non-S&E degree holders working in a non-S&E occupation.


Foreign-Earned S&E

Degree Holders


After April 2000

U.S. Earned S&E


Degree Holders


2001-2002 RCG (2003)

2003-2005 RCG (2006)

2006-2007 RCG (2008)

2008-2009 RCG (2010)


Degree Holders


2001-2002 SDR (2003)

2003-2005 SDR (2006)

2006-2007 SDR (2008)

2008-2009 SDR (2010)


Potential Design


Foreign-Earned S&E

Degree Holders

Prior to January 2009 NSCG

After January 2009 NSCG (2012-2018)

U.S. Earned S&E


Degree Holders

Prior to January 2009 NSCG

2009-2011 NSCG (2012)

2012-2013 NSCG (2014)

2014-2015 NSCG (2016)

2016--2017 NSCG (2018)


Degree Holders

Prior to January 2009 NSCG & SDR

2009-2011 NSCG & SDR (2012)

2012-2013 NSCG & SDR (2014)

2014-2015 NSCG & SDR (2016)

2016--2017 NSCG & SDR (2018)

25

Note: In the 2010 decade, SESTAT will provide coverage of all non-S&E degree holders regardless of

occupation.

Different SESTAT Data Focus –

Recent vs. Young

• The potential design would continue to cover the

recent college graduates population, but at a lower

precision level

• Under the potential design, the focus would change

from an emphasis on estimating recent college

graduates to estimating young college graduates

• To help us evaluate this focus change, we need to

understand how the current data is used and explore

the potential uses of data from the potential redesign

26

Exploration Question

• Can we leverage our use of the ACS-based NSCG to

derive NSCG estimates similar to those presently

provided by the NSRCG?

• As part of this exploration, we identified seven issues

that need detailed evaluation

27

Seven Issues to be Considered

1. The impact of the change in estimation concepts

(i.e., young graduates vs. recent graduates)

• “Recent graduates” refers to individuals earning a college

degree within the past two academic years whereas “young

graduates” refers to individuals below a particular age cutoff

without regard for the number of years since the graduation date.

• The ACS does not collect graduation date, so it is not possible to

target “recent graduates” in the ACS-based NSCG sample. The

ACS, on the other hand, does collect age, so it is possible to

target “young graduates” in the ACS-based NSCG sample

selection effort.

28

Seven Issues to be Considered

(continued)

2. Whether there is enough ACS sample to make this

switch

3. Accuracy of NSRCG vs. ACS-based estimates

4. Outreach to S&E and broader user community

5. Cost implications of NSRCG vs. ACS-based design

6. Impact on other NCSES surveys and data products such

as Women, Minorities and Persons with Disabilities and

Science and Engineering Indicators

7. Completion of evaluation summary report

29

Tentative Schedule Critical Evaluation Completion Date: March 2012

Ongoing Tasks

• Examine the sample availability

• Submission of Federal Register Notice

• Internal and external outreach to NSRCG community

• Examination of the cost implications

• Evaluate the precision of ACS-based estimates

Upcoming Tasks

• Continue outreach to NSRCG community to determine potential

impact on users

• Synthesize evaluation information and develop a summary

evaluation memorandum

• Review by CNSTAT panel 30

Impact on Users

• There will be no significant change in the coverage

of the S&E workforce from SESTAT under the

proposed design

• The SESTAT race and diversity questions will not

change, so the proposed changes will have NO

impact on the race and diversity data that will be

produced from SESTAT

• Long term, the proposed changes will allow

NCSES the ability to consider estimating the

Associate’s degree population

31

Assumptions

• The possibility of transitioning to an ACS-based

NSCG is based on the assumption of continued

funding of the ACS by the U.S. Census Bureau

• In addition, it is based on the assumption of the

continued availability of the ACS as a sampling

frame

• Any change to the funding or availability of the ACS

has severe implications on the feasibility of any

proposed SESTAT design

32

Data Needs and

Analytical Interests

Topic #1 – Recent vs. Young

• What survey is more appealing from a research

and/or policy perspective:

- A survey with a large number of recent

college graduates of all ages

- A survey with a large number of young adults

who are college graduates

34

Topic #2 – Research and Policy

Questions

• What are some of the important research and

policy questions we are answering through the use

of education/occupation statistics?

• What are some of the important research and

policy questions related to education/occupation

statistics for which no data source is available?

• What steps could be taken to address this

unavailable data source limitation?

35

Topic #3 – Emerging Issues

• What are some of the emerging estimation issues

in education statistics?

• Do you consider any of the following issues to be

of analytical interest? - Non-traditional graduates

- Professional master’s degree recipients

- The recent increase of master degree recipients

- The reasons for pursuing a master’s degree

- Community college enrollment

- Pathways ( or )

- S&E pipeline

36

Contact Information

37

John Finamore

National Center for Science and Engineering Statistics

National Science Foundation

[email protected]

(703) 292-2258

Reconsidering the Scientists and Engineers Statistical ...SESTAT • The NSF’s Scientists and...

Documents

Transcript of Reconsidering the Scientists and Engineers Statistical ...SESTAT • The NSF’s Scientists and...