Theme 3: Science of Science Policy to Address National NeedsSummaryDecember 4, 2008

High Level Questions

Total number of people who answered this section : 74Percent who rated high importancePercent who Ranked #1Question 8: What impact does science have on innovation and competitiveness?74.32%51.35%Question 9: How competitive is the US scientific workforce?43.24%21.62%Question 10: What is the relative importance of policy instruments in science policy?39.19%25.68%

Takeaway 1: Broaden the ScopeTheme implies that science serves only innovation- and competitiveness-oriented goals. However, science also serves broader societal goals (e.g., science for its own sake, environment, improvements in health, security, quality of life). The Roadmap should also reflect these goalsRoadmap is too US-centric, there is much happening internationally, we should be better networked and learn from others (e.g., data collection efforts in Europe and Asia)

Items on Which to VoteThe roadmap should clarify that economic competitiveness is not the sole goal of science

Science of science policy should have a stronger international component in its development phase

High Level Observation on Findings80% agreed with statement: The ITG finds that there is a real opportunity to develop new tools and data sets that could be used to quantify the impact of the scientific enterprise thus far on innovation and competitiveness (43% rated this number 1)

Equal split among other findings

High level observation on Recommendations

Top Models/ToolsKeeping in mind that we asked you to only comment on the tools you are familiar with, here are your ratings:

Peer/Expert Review #1Econometric; Cost/benefitData Collection: Survey Visualization tools rated very low

Top MetricsBibliometrics (citations; patents; scientific papers) highest ratedNothing else highly rated

Takeaway 2: Metrics/Methods/Tools are Meaningful Depending on ContextTools, methods and metrics need to be tied to levels (program, agency) and questions (what is the problem to be addressed); it is difficult to address them in abstract termsRelative weight of each depends on the question

Item on Which to Vote3. Usefulness of tools/metrics/measures should only be considered in the context of the specific question being posed

4. New data collection cannot be done willy-nilly: integrate into existing efforts, including better linkages and data mining

Takeaway 3: Need Better Articulation of ConceptsNational Needs need better articulationTough to talk about using SOSP to address national needs without explaining what they areSome terms must be cleaned up or better definedSay science and technology not just scienceDont conflate science with innovationTerm competitiveness meant different things to different people (science, economy)

*********