Competências Básicas de Investigação Científica e de Publicação

Ganesha Associates 2013

Competências Básicas de Investigação Científica e de Publicação

Lecture 8: Global science and the need for metrics

05/06/2013


Brasil’s economic growth creates a demand for scientists and technologists

05/06/2013


Science drives the economy• “The ability to create, distribute and exploit scientific and technological

knowledge has become a major source of competitive advantage, wealth creation and improvements in the quality of life.

• Some of the main features of this transformation are;– the growing impact of information and communications technologies (ICT) on the

economy and on society – the rapid application of recent scientific advances in new products and processes– a high rate of innovation across economically developed countries– a shift to more knowledge-intensive industries and services– rising skill requirements.”

• Source: Organisation for Economic Co-operation and Development ( OECD) Policy document 2000

05/06/2013

http://www.oecd.org/



Ganesha Associates 201305/06/2013

Global research investment

28 August 2012 Ganesha Associates 5


Document output in 2011

05/06/2013


Brasil:

05/06/2013

Ganesha Associates 2013Source: International Comparative Activity Performance – Elsevier 2011

BRIC output: documents

05/06/2013


Citation quality is a problem

05/06/2013


International collaboration is low

05/06/2013


Share of articles by co-authorship type

05/06/2013

EPI : English Proficiency Index by age


Institutional ranking

05/06/2013


• O Qualis é um levantamento realizado pela Capes com o intuito de mensurar a qualidade da produção científica dos programas de pós-graduação.

• A classificação de periódicos é realizada pelas áreas de

avaliação e passa por um processo anual e atualização. Esses veículos são enquadrados em estratos indicativos de qualidade; A1, o mais elevado; A2; B1; B2; B3; B4; B5; C, com peso zero.

• O seu programa de pós-graduação será avaliado segundo essa escala.

05/06/2013

Brasil Qualis and the importance of Indexação

UK’s Research Excellence Framework 2014

• Outputs: The panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards. This element will carry a weighting of 65 per cent.

• Impact: The sub-panels will assess the ‘reach and significance’ of impacts on the economy, society and/or culture that were underpinned by excellent research conducted in the submitted unit, as well as the submitted unit’s approach to enabling impact from its research. This element will carry a weighting of 20 per cent.

• Environment: The sub-panels will assess the research environment in terms of its ‘vitality and sustainability’, including its contribution to the vitality and sustainability of the wider discipline or research base. This element will carry a weighting of 15 per cent.

Global metrics summary

• Science and technology are too politically important for them to be left un-managed

• The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form

• Metrics will become more important as better standards are defined

Portfolio analysis at NIH

• What is the state of the science? • Are there gaps in our research portfolio?• How do the gaps compare to research carried

out in other agencies?• How much do we spend on mitochondrial

research?• What is the average number of scientific

papers generated per $100K spent per grant?


Define: indexação

• A indexação de uma revista é o processo que confere o indicador de qualidade necessário para que esta possa pertencer a um ou mais bancos de dados de renome internacional, em uma área específica do conhecimento, como por exemplo, MEDLINE, ISI, LILACS, etc., segundo critérios preestabelecidos por essas instituições.

• É um fator importante para que a produção científica nela apresentada se torne conhecida e reconhecida internacionalmente.

• O periódico necessita, para isto, estar de acordo com os padrões básicos de apresentação formal e excelência de conteúdo científico, determinados internacionalmente.


As fontes de‘indexação’ em ordem de ‘qualidade’

• Thomson Web of Science• PubMed/Medline• Scopus• ScieLo• Google• Scirus• LILACS


Journal selection process - ISI

• To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well.

• However, rather than compare a regional journal with all journals in a particular subject category, the ISI editor considers it in terms of those journals in the category from the same geographic area.

• High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential.


Impact Factor defined

“The Impact Factor of a journal is probably the only quantitative way of assessing its worth and relevance to the academic community it serves.” Editorial, Journal of Computer-assisted Learning, 2001

Impact Factor 2011 =# citations received in 2011 to articles published in 2010/2009______________________________________________________

# number of source items published 2010 and 2009

Journal of Cell Science - IF

Nature citations per article

Citation statistics – PLoS Biology

5 July 2012 Ganesha Associates 35

Ganesha Associates 3614 August 2013


Impact Factors over time


Impact Factor and area


Impact Factors for Microbiology


Impact Factors for Neuroscience


Impact Factors and article type


Impact Factors and Authors

Correlation between article citation rate and journal impact for four authors


Impact Factors and Journals

Citation rates in 1986 or 1987 of articles published in three biochemical journals in 1983 or 1984, respectively. From Seglen, BMJ 1997;314:497


Alternatives ?• The Hirsch Index is rapidly gaining favor as a

performance measurement tool for scientific authors.

• A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np - h) papers have no more than h citations

• Thus, the h-index reflects both the number of publications and the number of citations per publication. The index is designed to improve upon simpler measures such as the total number of citations or publications.

• Like the IF, the index works properly only for comparing scientists working in the same field.

• For more information see Scopus or Wikipedia


H-index - weaknesses• The h-index is bounded by the total number of

publications. • This means that scientists with a short career are at

an inherent disadvantage, regardless of the importance of their discoveries.

• For example, had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date.

Declaration on Research Assessment

• Made during the Annual Meeting of The American Society for Cell Biology (ASCB) in San Francisco, CA, on December 16, 2012.

• Main recommendation: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

• Alternatives?

New Usage Metrics• Impact Factor

Well-established, easily understood and acceptedEndorsed by funding agencies and researchersSeveral competitors: H-index, Eigenfactor, SJR (Google Page Rank)

• COUNTER Usage FactorJournal/book usage-based alternative perspectiveReflects value of journals to all categories of userEasy to understand

• PIRUS (Publisher and Institutional Repository Usage Statistics) COUNTER-compliant usage reports at the article levelWill enable the usage of individual research outputs to be measured

• ORCID (Open Researcher and Contributor ID)Central registry of identifiers from individual researchersWill enable aggregation of multiple forms of research output

FundRef

Altmetrics, an alternative?

5 July 2012 Ganesha Associates 49


Many metrics, many uses…• Development of national-level social contracts for science and technological investment• Does investment lead to the achievement of national/international objectives for

increased employment, economic growth, improved healthcare? • At a national/institutional/departmental/individual level how do we compare with the

competition?• Management of human resources and tacit intellectual property in a multinational

organisation• Do I get tenure/promotion?• How often has my article been downloaded/cited• How well has my paper contributed to building the knowledge network in my field?• Which journal should I publish in?• Which articles should I read?• Has my article been picked-up by the media?

• Standards: CrossRef, ORCID, DataCite, FundRef

14 August 2013

Snowball metrics

Global metrics summary

• Science and technology are too politically important for them to be left un-managed

• The outputs of science and technology are too numerous and complex for them to be managed without reference to metrics of some form

• Metrics will become more important as better standards are defined

Competências Básicas de Investigação Científica e de Publicação

Documents

Transcript of Competências Básicas de Investigação Científica e de Publicação