5. ROADMAP 5. ROADMAP 5.6 he digitalisation of scienceT · 1 MEASURING THE DIGITAL TRANSFORMATION:...
2
MEASURING THE DIGITAL TRANSFORMATION: A ROADMAP FOR THE FUTURE © OECD 2019 1 5. ROADMAP Why are indicators on the digitalisation of scientific research needed? Ministers from OECD countries and partners meeting at the OECD Ministerial Meeting held in Daejeon (Korea) in 2015 recognised in their joint declaration (www.oecd.org/sti/daejeon-declaration-2015.htm) that science, technology and innovation (STI) are being revolutionised by the rapid evolution of digital technologies. These technologies are changing the way in which scientists work, collaborate and publish; increasing the reliance on access to scientific data and publications; opening new avenues for public engagement and participation in science and innovation; facilitating the development of research co-operation between businesses and the public sector; and contributing to the transformation of innovation. The OECD was asked to monitor this transformation and invited to convene the international community working on STI data and indicators to develop new thinking and solutions for empirical evidence to guide policy. The 2016 OECD Blue Sky Forum (http://oe.cd/blue-sky) identified the digitalisation of STI both as a priority object of measurement and as a fundamental enabler of future statistical and analytical work (OECD, 2018). What are the challenges? Tracing the resources invested in expanding knowledge of the possibilities of digitalisation and their outputs requires data infrastructures with a wider scope that provide high levels of detail. Research domains evolve over time and new paradigms emerge that draw upon existing research. Conventional practices and categories, though necessary, are not sufficient to track the digitalisation of research and its impact on digital transformation. New text- mining tools enable the extraction of relevant information from qualitative sources to produce indicators about the nature, method and purpose of research. Provided the underlying data are sufficiently rich and comprehensive and the use of the mining techniques is sound, it is possible to estimate with some accuracy the share of scientific production related to topics such as AI or the percentage of funding agency support for R&D projects that makes use of AI tools. However, the databases that can in principle support this type of analysis are often fragmented, difficult to gain access to, and challenging to combine and use. Data users increasingly demand access to a very fine-grained level of detail that can be hard to reconcile with the preservation of confidentiality and, depending on the way in which data are used to inform decisions, may induce selective disclosure of information by individuals or organisations, if this is in their interest. Because project funding and career decisions are influenced by quantitative indicators, a roadmap for science measurement in the digital age has to address the potential trade-off between reconciling data availability and integrity arising from exploiting new sources. Another key challenge is to connect measures of science digitalisation to policy measures. Options for international action Initiatives to leverage information captured by digital systems to measure both scientific research and its digital transformation has been supported by consortia of independent non-profit organisations, academia and business that promote the growing adoption of standards. These standards target digital objects that capture multiple dimensions of scientific activity, such as documents, projects, data and code, allowing them to be persistently identified and retrieved, and also enabling them to be interpreted, linked or attributed to individuals and their organisations. The ORCID identifier (https://orcid.org) is a salient example of the new data infrastructure about science and research, having attained high adoption rates among serving researchers, their organisations, and many publishers and research funders. Although the main motivation for STI data infrastructures with such features is to facilitate scientific and related management processes rather than statistics, these data can provide the basis for higher quality statistics and indicators. Administrative reporting requirements often set out the basis for what official surveys can collect from respondents. Different groups within the science and science management community can work to ensure the integrity of the information, but there are significant career and commercial interests at play, as well as co-ordination challenges. Because of the current limitations of available sources of information about some rapidly evolving dimensions of science, the OECD has been experimenting with the development of a dedicated survey tool. The OECD International Study of Scientific Authors (ISSA) is a new global survey-based initiative that was first piloted in 2015 to explore the changing scientific publishing landscape; its 2018 edition examined how science is going digital. ISSA explores a range of key dimensions that are potentially relevant across all fields of science: 5.6 The digitalisation of science
Transcript of 5. ROADMAP 5. ROADMAP 5.6 he digitalisation of scienceT · 1 MEASURING THE DIGITAL TRANSFORMATION:...
MEASURING THE DIGITAL TRANSFORMATION: A ROADMAP FOR THE FUTURE © OECD 20191
5. ROADMAP 5. ROADMAP
Why are indicators on the digitalisation of scientific research needed? Ministers from OECD countries and partners meeting at the OECD Ministerial Meeting held in Daejeon (Korea) in 2015 recognised in their joint declaration (www.oecd.org/sti/daejeon-declaration-2015.htm) that science, technology and innovation (STI) are being revolutionised by the rapid evolution of digital technologies. These technologies are changing the way in which scientists work, collaborate and publish; increasing the reliance on access to scientific data and publications; opening new avenues for public engagement and participation in science and innovation; facilitating the development of research co-operation between businesses and the public sector; and contributing to the transformation of innovation. The OECD was asked to monitor this transformation and invited to convene the international community working on STI data and indicators to develop new thinking and solutions for empirical evidence to guide policy. The 2016 OECD Blue Sky Forum (http://oe.cd/blue-sky) identified the digitalisation of STI both as a priority object of measurement and as a fundamental enabler of future statistical and analytical work (OECD, 2018).
What are the challenges?Tracing the resources invested in expanding knowledge of the possibilities of digitalisation and their outputs requires data infrastructures with a wider scope that provide high levels of detail. Research domains evolve over time and new paradigms emerge that draw upon existing research. Conventional practices and categories, though necessary, are not sufficient to track the digitalisation of research and its impact on digital transformation. New text- mining tools enable the extraction of relevant information from qualitative sources to produce indicators about the nature, method and purpose of research. Provided the underlying data are sufficiently rich and comprehensive and the use of the mining techniques is sound, it is possible to estimate with some accuracy the share of scientific production related to topics such as AI or the percentage of funding agency support for R&D projects that makes use of AI tools. However, the databases that can in principle support this type of analysis are often fragmented, difficult to gain access to, and challenging to combine and use.
Data users increasingly demand access to a very fine-grained level of detail that can be hard to reconcile with the preservation of confidentiality and, depending on the way in which data are used to inform decisions, may induce selective disclosure of information by individuals or organisations, if this is in their interest. Because project funding and career decisions are influenced by quantitative indicators, a roadmap for science measurement in the digital age has to address the potential trade-off between reconciling data availability and integrity arising from exploiting new sources. Another key challenge is to connect measures of science digitalisation to policy measures.
Options for international actionInitiatives to leverage information captured by digital systems to measure both scientific research and its digital transformation has been supported by consortia of independent non-profit organisations, academia and business that promote the growing adoption of standards. These standards target digital objects that capture multiple dimensions of scientific activity, such as documents, projects, data and code, allowing them to be persistently identified and retrieved, and also enabling them to be interpreted, linked or attributed to individuals and their organisations. The ORCID identifier (https://orcid.org) is a salient example of the new data infrastructure about science and research, having attained high adoption rates among serving researchers, their organisations, and many publishers and research funders. Although the main motivation for STI data infrastructures with such features is to facilitate scientific and related management processes rather than statistics, these data can provide the basis for higher quality statistics and indicators. Administrative reporting requirements often set out the basis for what official surveys can collect from respondents. Different groups within the science and science management community can work to ensure the integrity of the information, but there are significant career and commercial interests at play, as well as co-ordination challenges.
Because of the current limitations of available sources of information about some rapidly evolving dimensions of science, the OECD has been experimenting with the development of a dedicated survey tool. The OECD International Study of Scientific Authors (ISSA) is a new global survey-based initiative that was first piloted in 2015 to explore the changing scientific publishing landscape; its 2018 edition examined how science is going digital. ISSA explores a range of key dimensions that are potentially relevant across all fields of science:
5.6 The digitalisation of science
MEASURING THE DIGITAL TRANSFORMATION: A ROADMAP FOR THE FUTURE © OECD 2019 2
5. ROADMAP 5. ROADMAP
The use of digital tools in support of scientific work
(e.g. search, collaboration,communication).
The role of data and ICTin research methods,
identifying to what extentscience is enabled or
driven by new data sourcesand data explorationopportunities such
as Big data.
The generation byscientists of accessibledigital products such as
data and code.
The digital footprintof scientist’s work and
its role in scholarlycommunication,
research assessment,incentives and research
careers.
Scientists’ views on theimpacts of digitalisation, in
terms of research efficiency,efficacy and inclusiveness
The ISSA 2018 results show that the digitalisation of science is not limited to scientific fields that specialise in computer science or IT engineering. They also indicate significant potential for greater IT adoption in general scholarly practice, as well as for harnessing the potential of data-driven research. The ISSA 2018 survey highlights potential limits to the adoption of the digital footprint of research as a basis for new science indicators.
Use and development of Big data across scientific domains, 2018
0
20
40
60
80
100% Only use Develop Neither use nor develop
Compu
ter Scie
nce
Chemica
l Eng
ineeri
ng
Genera
l
Energ
y
Engin
eerin
g
Agricu
ltural
and B
iolog
ical S
cienc
es
Earth
and P
laneta
ry Scie
nces
Bioche
mistry,
Geneti
cs an
d Mole
cular
Biolog
y
Materia
ls Scie
nce
Envir
onmen
tal Scie
nce
Physic
s and
Astron
omy
Econ
omics
, Eco
nometr
ics an
d Fina
nce
Medici
ne
Mathem
atics
Busine
ss, M
anag
emen
t and
Accou
nting
Pharm
acolo
gy, T
oxico
logy a
nd Pha
rmac
eutic
s
Chemist
ry
Social
Scienc
es
Arts an
d Hum
anitie
s
Psych
ology
Source: OECD International Survey of Scientific Authors (ISSA), 2018, preliminary results, http://oe.cd/issa, January 2019. See chapter notes.12 https://doi.org/10.1787/888933930459
The experience of the ISSA study confirms several of the challenges of conducting surveys in the digital age, especially when trying to ensure trust between data collector and respondent. The ISSA survey is ultimately an exploration mechanism aimed at developing working knowledge on emergent topics of high policy relevance. This can help provide a potential basis for distributed data collection within countries and a mechanism for ongoing dialogue between the OECD and the global science community.
Reference
OECD (2018), OECD Science, Technology and Innovation Outlook 2018: Adapting to Technological and Societal Disruption, OECD Publishing, Paris, https://doi.org/10.1787/sti_in_outlook-2018-en.
5.6 The digitalisation of science
