The impact of Monte Carlo simulation: a scientometric analysis of scholarly literature

Maria Grazia Pia, INFN Genova

The impact of Monte Carlo simulation: a scientometric analysis of scholarly literature

Maria Grazia Pia INFN Genova, Italy

Maria Grazia Pia1, Tullio Basaglia2, Zane W. Bell3, Paul V. Dressendorfer4

1INFN Sezione di Genova, Italy – 2CERN, Switzerland – 3ORNL, USA – 4IEEE, USA

SNA + MC 2010Joint International Conference on

Supercomputing in Nuclear Applications + Monte Carlo 2010


To write or not to write?

T. Basaglia, Z. W. Bell, P. V. Dressendorfer, A. Larkin, M.G. Pia, Writing Software or Writing Scientific Articles? IEEE Trans.

Nucl. Sci., vol. 55, no. 2, pp. 671-678, 2008

Software publications are largely underrepresented in

particle/nuclear physics literature w.r.t. hardware ones

IEEE NSS 2007

However, some software publications collect a large number of citations

S. Agostinelli et al.,Geant4 - a simulation toolkitNIM A, vol. 506, pp. 250-303, 2003

2079 citations18 October 2010


Features of this study

Thomson-Reuters, ISI Web of Knowledge Web of Science, Journal Citation Reports INFN subscription: since 1990 CERN subscription: since 1970

Journal web sites Not all publishers provide full-text search capabilities

Automated analysis Whenever possible

Manual scan of publication records, abstracts and full-text papers Some degree of subjectivity

Sample of representative journals in research areas exploiting Monte Carlo simulation Does not cover the whole scope of physics/engineering literature


Not just one champion paper…

58% TNS45% NIM A+Bpapers mention

“simulation” or “Monte Carlo”

In the last 5 years (2004-2009)

1960 1964

1965 1969

1970 1974

1975 1979

1980 1984

1985 1989

1990 1994

1995 1999

2000 2004

2005 2009

0100020003000400050006000700080009000

10000110001200013000

TNS NIM A+B Nucl Phys BPhys Lett B Phys Rev D Phys Rev LettMed Phys PMB APJ

Years

Oc

cu

rre

nc

e

The number of papers mentioning Monte Carlo or simulation has been increasing in the past 50 years

1960 1964

1965 1969

1970 1974

1975 1979

1980 1984

1985 1989

1990 1994

1995 1999

2000 2004

2005 2009

0

10000

20000

30000

40000

50000

60000

70000

80000

TNS NIM A+B Nucl Phys BPhys Lett B Phys Rev D Phys Rev LettMed Phys PMB APJ

Years

Pu

bli

shed

pap

ers

Also the total number of published papers has increased


Patterns in TNS papers~ 60% recent publications mention model*

~ 60% recent publications mention Monte Carlo OR simulationThe fraction of papers mentioning these words has been growing

over the last 50 years

The increase appears to be associated with Monte Carlo and/or

simulation

Roughly constant fraction of papers mentioning

model*, but not mentioning

Monte Carlo or simulation19

6019

6319

6619

6919

7219

7519

7819

8119

8419

8719

9019

9319

9619

9920

0220

0520

080.0

0.1

0.2

0.3

0.4

0.5

0.6

MC/simulation model*model* AND MC/simulation model NOT MC/simulation

Year

Fra

cti

on

of

pu

blis

he

d p

ap

ers


Technological journals

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

0.0

0.1

0.2

0.3

0.4

0.5

0.6

Monte Carlo OR simulation

NIM A+BTNSNIM ANIM B

Year

Fra

cti

on

of

pu

bli

sh

ed

pa

pe

rs

Similar fraction of papers mentioning Monte Carlo or simulation in NIM and TNS until ~1990; then larger fraction in TNS, but similar

trend of increasing fraction

1960-2009


1960 1964

1965 1969

1970 1974

1975 1979

1980 1984

1985 1989

1990 1994

1995 1999

2000 2004

2005 2009

0%

5%

10%

15%

20%

25%

Phys Rev D Phys Rev LettNucl Phys B Phys Lett B

Years

Oc

cu

rre

nc

e

1960 1964

1965 1969

1970 1974

1975 1979

1980 1984

1985 1989

1990 1994

1995 1999

2000 2004

2005 2009

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Med Phys PMB

Years

Oc

cu

rre

nc

e

Physics journals Medical physics journals

Monte Carlo or Simulation


Relative presence in literature

1960 1964

1965 1969

1970 1974

1975 1979

1980 1984

1985 1989

1990 1994

1995 1999

2000 2004

2005 2009

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

TNS Phys Rev D Phys Rev LettMed Phys PMB APJNIM A NIM B Nucl Phys BPhys Lett B

Years

Oc

cu

rre

nc

e

Monte Carlo or Simulation


What contributes to the increasing popularity of Monte Carlo simulation?

Analysis in progressSocio-economic factors, computing facilities,

availability of general purpose codes

1960

1964

1968

1972

1976

1980

1984

1988

1992

1996

2000

2004

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

50000

0.00

0.10

0.20

0.30

0.40

0.50

0.60

US TNS MC/sim

Year

Inco

me

($)

Fra

ctio

n o

f M

C/s

imu

lati

on

pap

ers

1970.011976.011982.011988.011994.012000.012006.010

5000

10000

15000

20000

25000

30000

35000

40000

45000

0.1

0.2

0.3

0.4

0.5

0.6

Nikkei 225 TNS

Year

Nik

kei

225

Fra

ctio

n o

f p

aper

s

The butterfly effect, IEEE NSS 2010


Monte Carlo codes in technological research

Fraction of papers mentioning well-known Monte Carlo codes

GEANT 3

Geant4

EGS

FLUKA

MCNP

Penelope

0% 2% 4% 6% 8%

NIMTNS

~ 9%~ 15%

Papers published in 2004-2009

NIM 13407 papers TNS 2630 papers


Monte Carlo enables physics

Phys. Rev. D

Phys. Rev. Lett.

Phys. Rev. C

Phys. Rev. ST Accel.

Phys. Rev. E

Phys. Rev. B

Phys. Rev. A

0 100 200 300 400 500 600

PenelopeFLUKAMCNPEGSGeant4GEANT 3

APS journals, 1990-2008

Full-text search in http://prola.aps.org/

Papers mentioning well-known Monte Carlo codes

Number of citations

http://prola.aps.org/


GEANT 3

Geant4

EGS

FLUKA

MCNP

Penelope

0 50 100 150 200 250

Phys. Rev. CPhys. Rev. Lett.Phys. Rev. D

2004-2008GEANT 3 still widely used in

physics production (CDF, D0, CLEO, BES, Belle etc.)

Geant4 and MCNP more often mentioned in

technological journals,GEANT 3 appears less popular in technology

research

2004-2008

GEANT 3

Geant4

EGS

FLUKA

MCNP

Penelope

0 50 100

150

200

250

300

350

400

450

500

NIMTNS

Mostly GEANT 3.21, but also older versions

Not only Geant4!

Cutting edge of obsolescence

Number of citations

Number of citations


The most cited paper

The most cited paper of the whole

Nuclear Science & Technologycategory is about a Monte Carlo code

Source: Thomson-Reuters ISI Web of Knowledge

Period: 1970-2010

3rd most cited CERN paper2nd most cited INFN paper

2nd most cited physics paper in Japan(excluding Review Part. Phys.)

stay tuned… still growing!

http://apps.isiknowledge.com/


Geographical distribution

N America

Europe

Russia+

Asia

S America

Oceania

Africa

0% 10% 20% 30% 40% 50% 60% 70% 80%

Citations to Geant4NIM 2003 paper

All

Excluding BaBar

2003-2010 citations

Top 10

USA

GERMANY

ITALY

FRANCE

ENGLAND

RUSSIA

SPAIN

CANADA

NETHERLANDS

SCOTLAND

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Country

Update:18 October 2010

USA

GERMANY

ITALY

SWITZERLAND

FRANCE

ENGLAND

JAPAN

RUSSIA

SPAIN

CANADA

0% 5% 10% 15% 20% 25% 30% 35% 40%

BaBar: 231 papers


Citing institutes

All

Excluding BaBar Excluding BaBar and CERN

2003-2010 citations

Geant4 NIM 2003 paper

Top 10Update:18 October 2010

INFN

UNIV VALENCIA

UNIV ROMA 1

RAL

UNIV PADUA

UNIV LIVERPOOL

OHIO STATE UNIV

UNIV MILAN

UC BERKELEY

HARVARD UNIV

0% 5% 10% 15% 20% 25%

INFN

CERN

UNIV VALENCIA

UNIV TOKYO

JINR

KYOTO UNIV

UNIV OXFORD

RUSSIAN ACAD SCI

UCL

UNIV ROMA 1

0% 2% 4% 6% 8% 10% 12% 14%

INFN

UNIV TOKYO

KYOTO UNIV

JINR

UNIV VALENCIA

CHINESE ACAD SCI

NASA

UNIV LIVERPOOL

RIKEN

FERMILAB

0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10%


Citing journals

• Technology w/o conf. proc.• HEP w/o conf. proc.• Nuclear Physics w/o proc.• Medical Physics w/o proc.• Astroparticle Physics w/o

proc.

Top 10

Wide scope, including: Phys. Rev. A/B/C/D/E Anal. Chem. Geophys. Res. Lett. Plasma Sci. Technol. Appl. Surf. Sci. Appl. Eng. Agriculture etc.

Main source of citations: HEP and technology

journals


NIM A

IEEE TNS

Med Phys

Phys Rev C

Astropart Phys

0% 2% 4% 6% 8%10%

12%14%

16%18%

20%

Update:18 October 2010


HEP citations

BaBar

LHC

CERN non-LHC

CDF

BES

Astroparticle

Linear Collider

Other

0% 10% 20% 30% 40% 50%

Semi-automated classification: authors (ISI Web)

experiment identification (ISI Web)

manual inspection

Plot → using Geant4 + producing archival results + publishing + citing


59% BaBar physics papers (2004-2008)


2002 2003 2004 2005 2006 2007 2008 2009 2010 20110

50

100

150

200

250

300

350

400

450

Citations to Geant4 NIM 2003

Year

Cit

ati

on

sFluctuation or trend?

How will publications by LHC experiments affect the picture in the next years?

And other disciplines?

Scaled to end 2010

ATLAS 6NIM A 5EPJC 1

CMS 4NIM A 2JHEP 1J Phys G 1

LHCB 2 NIM A 2

ALICE 1 NIM A 1

LHC experiments 2009-2010Citations to Geant4 NIM 2003

2002 2003 2004 2005 2006 2007 2008 2009 2010 20110

5

10

15

20

25

30

35

40

45

BaBar experiment citations to Geant4 NIM 2003

Year

Cit

ati

on

s

Experimental life-cycle

Mostly by groups


Phys. Rev. D

Phys. Rev. Lett.

Phys. Rev. C

TNS

NIM A+B

0 50 100 150 200 250 300 350 400 450

CiteMention

59%NIM 2003 reference

Missing citations2004-2008 publications

Full-text search in publishers’ webs

82%

93%

64%%

51%

27% TNS papers published in 2007-2008 mentioning Geant4 and 10% NIM A+B ones cite the TNS 2006 reference

articles mentioning Geant4 cite the NIM A 2003 reference

40% Elsevier

81% APS54% NIM-TNS


ConclusionsMajority of technological literature reports use of simulation and Monte Carlo

Monte Carlo plays a major role in producing physics results

Use of Monte Carlo codes is increasing

Physics community still heavy user of older code GEANT 3

HEP and Medical Physics researchers are the dominant users of Geant4 Significant presence of astroparticle

community LHC: now running, how will the citation

statistics evolve ?

Many Monte Carlo users do not cite reference for code used in their papers

The impact of Monte Carlo simulation: a scientometric analysis of scholarly literature

Documents

Transcript of The impact of Monte Carlo simulation: a scientometric analysis of scholarly literature