Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in...
-
Upload
ferdinand-greene -
Category
Documents
-
view
212 -
download
0
Transcript of Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in...
![Page 1: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/1.jpg)
Strategic Evaluation of Research
–Viewpoint of Outcome in the Evaluation of Research Units
in AIST–M. Daté, H. Tokunaga, S. Nakatsu, S. Ito,
H. Matsuhata, H. Ogi, A. Omori, S. Suto, M. Tajima, T. Tanaka, K. Urabe, O. Nakamura*, O. Nakamura, S. Kosaka, M. Takagi Sawada and N. Kobayashi
Natl. Inst. Adv. Ind. Sci. Technol. (AIST), JAPAN*Min. Econ. Trade Ind. (METI), JAPAN
1/15
![Page 2: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/2.jpg)
Outline of AISTNew AIST (National Institute of Advanced Industrial Science and Technology) was established in 2001 as an Independent Administrative Institute.
ResearchersAdministrative StaffsTotal Employees
2,505704
3,209 As of April 2006
2/15
![Page 3: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/3.jpg)
Mission of AISTBasic Principle is “Realization of a Sustainable Society”.
3/15
![Page 4: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/4.jpg)
Research Fields in AIST
4/15
![Page 5: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/5.jpg)
Organization of AIST
As of August 2006
5/15
Collaboration Dept
Auditor
President
Research Centers
Research Institutes
Research Initiatives
Senior Vice-PresidentVice-President
Emeritus Advisor
AIST Advisory Board
Research Coordinator
Emeritus Councilor
AIST Fellow
Research Units
adm
inis
trat
ive
un
its
Intellectual Property Dept
Technology Information Dept
International Affairs Dept
General Administration Dept
Human Resources Dept
Financial Affairs Dept
Research Facilities Dept
Intl Patent Organism Depositary
AIST Innov Center for Start-ups
Geo-information Center
Metrology Management Center
Tsukuba Adv Comp Center (TACC)
Planning HQs
General Administration HQs
Evaluation Dept
Safety & Environ Protection Dept
Public Relations Dept
Legal Office
Info Disclosure & Personal Info Protection Promotion Office
Gender Equality Office
Audit Office
![Page 6: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/6.jpg)
Research Units of AIST
Research Centers (29)
Priority in investing research resources
Operated by top-down management
Pioneering and strategic projects within a certain period (<7 years)
Research Institutes (21)
Proposal of research subjects in a bottom-up manner
Mid- and long-term researches
Research Initiatives (7)
Response to sudden administrative needs
Aiming for new research centers or fields within a short period (<3 years)
As of August 2006
6/15
![Page 7: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/7.jpg)
Evaluation of Research Units
Purposes of evaluation
2) Reference to the top management of AIST
3) Accountability to the Japanese Nation
1) Activation and efficient management of Research Units
1st Mid-Term (FY2001-2004) 2nd Mid-Term (FY2005-2009)
Evaluation based on Outputs and global level of research
Evaluation from the viewpointof Outcome
Viewpoint of evaluation
7/15
![Page 8: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/8.jpg)
Definition of Terms in AISTOutputs:
Direct results in R&D (e.g., papers, patents, etc.)
・・ BudgetBudget・・ Human Human resourceresource・・ EquipmentEquipment
・・ Academic Academic paperpaper・・ PatentPatent・・ Geologic MapGeologic Map・・ StandardStandard・・ SoftwareSoftware・・ Process Develop.Process Develop.・・ DatabaseDatabase
・・ Popularization Popularization of productsof products・・ New research New research fieldfield・・ ServiceService・・ Saving energy Saving energy ・・ Risk Risk managementmanagement・・ DisasterDisaster preventionprevention
INPUT IMPACTOUTPUT OUTCOME
・・ GDP growthGDP growth・・ New employmentNew employment・・ InternationalInternational competition abilitcompetition abilityy・・ High QoLHigh QoL・・ Environmental Environmental protectionprotection・・ Security & safety Security & safety societysociety
Outcomes:Social and economical effects caused by outputs (e.g., products, standards, novel research fields, etc.)
8/15
![Page 9: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/9.jpg)
Evaluation System in AIST
Outputs in the past Realized Outcomes
Expected Outcomesin the future
1) Roadmaps2) Outputs3) Management
PAST PRESENT FUTURE
Subjects of evaluation
2) Evaluation of Outputs toward Outcomes
3) Evaluation of Management toward Outcomes
1) Evaluation of Roadmaps toward Outcomes
9/15
![Page 10: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/10.jpg)
Evaluation IndexesEvaluation of Roadmaps
Outcomes, Milestones, Technical Elements, Benchmarks
Evaluation of Outputs
Papers, Patents, Standards, Academic Activities, Press Releases, Prizes, Prototypical Products, etc.
Evaluation of Management
Research Strategy
Other Management
1) Full Research, 2) Innovations, 3) Human Resource Development
1) Overall Management, 2) Budget Management, 3) Risk Management
10/15
![Page 11: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/11.jpg)
Reviewers in EvaluationExternal Reviewers (5-7 for each Research Unit)
e.g., academia, industry, journalism, etc.
Specialists and intellectuals outside of AIST
Internal Reviewers (3 for each Research Unit)
Principal Reviewers of AIST, etc.
Main Role: Evaluation of Roadmaps and Outputs
Main Role: Evaluation of Management
(Roadmaps & Outputs) : Management = 7 : 3
Allocation of grades
11/15
![Page 12: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/12.jpg)
Evaluation Procedures
Resource, ReorganizationResource, Reorganization
Evaluation Committee
Evaluation Committee
1) Roadmaps2) Outputs3) Management
Evaluation DepartmentEvaluation Department
Activity Report
Presentation
Activity Report
PresentationComments on Evaluation
Claim of Reconsideration
Comments on Evaluation
Claim of Reconsideration
Grades Comments
Grades Comments
CorrectionCorrection
External ReviewersInternal Reviewers
External ReviewersInternal Reviewers
Research UnitResearch Unit
12/15
Evaluation Committeeof Ind. Admin. Inst.Evaluation Committeeof Ind. Admin. Inst.
PresidentPresidentJapanese NationJapanese Nation
Research Strategy of AISTResearch Strategy of AIST
Overall Evaluation Overall Evaluation
![Page 13: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/13.jpg)
Survey of the Unit Directors (1)
Was the evaluation of this year useful?Was the evaluation of this year useful?
Was the evaluation worth all the efforts?Was the evaluation worth all the efforts?
FY2005
FY2005
FY
FY
13/15
![Page 14: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/14.jpg)
Survey of the Unit Directors (2)
Was the viewpoint of outcomes useful?Was the viewpoint of outcomes useful?
Was the evaluation useful in improving management?Was the evaluation useful in improving management?
FY2005
FY2005
FY
FY
14/15
![Page 15: Strategic Evaluation of Research –Viewpoint of Outcome in the Evaluation of Research Units in AIST– M. Daté, H. Tokunaga, S. Nakatsu, S. Ito, H. Matsuhata,](https://reader036.fdocuments.us/reader036/viewer/2022070401/56649f1b5503460f94c30b85/html5/thumbnails/15.jpg)
Comments of External ReviewersCATEGOLY PERCENTAGE
Affirmative comments on the evaluation system
20
Comments on the evaluation of roadmaps
14
Misunderstanding, insufficient explanation
14
Insufficient time or activity reports
12
Proposal to improve the evaluation system
10
Request for various evaluation indexes 9
Request for a longer interval of evaluation
8
Request to reduce the load of evaluation
6
Other comments 9
FY 2006
15/15