SYSTEMATIC REVIEW ACCELERATOR:

THE POLYGLOT SEARCH TRANSLATOR

Justin Clark

Centre for Research in Evidence Based

Practice (CREBP)

Bond University, Gold Coast, Australia

SR tasks – full list of tasks and toolsTask No. Classification Task Description Automation tool

1 Preparation Formulate review

question

Decide on the research question of the

review.

SRA – TRIP data analysis

2 Preparation Find previous or

upcoming SR

Search for a SR that answers the same

question.

Cochrane library, Epistemonikos, PROSPERO

3 Write up Write the protocol Provide an objective, reproducible, sound

methodology for peer review.

Reporting guidelines, existing protocol templates

4 Preparation Devise search strategy Decide on databases and keywords to find

all relevant trials.

SRA - Registered Trials Finder (in development), SRA - Word Frequency

Analyser

5 Retrieval Search Aim to find all relevant citations even if

many irrelevant ones included.

SRA - Polyglot Search Translator

6 Retrieval De-duplicate Remove identical citations. SRA - De-duplicator. Rayyan

7 Appraisal Screen abstracts Based on titles and abstracts, remove

definitely irrelevant citations.

SRA - Endnote Helper, Covidence, Rayyan

8 Retrieval Obtain full text Download, request copies from authors,

inter-library loan, etc.

Endnote - find full text feature, SRA – Endnote Helper

9 Appraisal Screen full text Exclude irrelevant citations. SRA - Endnote Helper, Covidence

10 Retrieval Citation analysis Follow citations, cited and citing, from

included trials to find additional trials.

SRA - SpiderCite (in development)

11 Synthesis Extract data Extract outcome numbers and associate

with trial arm

Robot Reviewer

12 Synthesis Synthesize data Convert extracted data to common

representation (usually average and SD).

Revman

13 Retrieval Recheck literature Repeat the search to find new literature

published since the initial search.

SRA - Polyglot Search Translator (future development)

14 Synthesis Meta-analyze Statistically combine the results from all

included trials.

RevMan, MetaXL

15 Write up Write up review Produce and publish the final report SRA – Revman Replicant

Polyglot Search Translator (PST)

• The PST translates either a PubMed or Ovid Medline

search strategy to the correct syntax for different

databases

Polyglot Search Translator

• Copy your PubMed or Ovid Medline search and paste it in to the ‘Your Query’

box

• To see how the search translates across databases, click the arrow next to

the name of the database.

Polyglot Search Translator

• For some databases you will need to convert the subject terms (e.g. change

MeSH terms to Emtree terms)

Polyglot Search Translator

• The PST is designed to make searching databases easier, BUT it does not

remove the need to have knowledge of the process

Medline Ovid to Cochrane Central

Evaluation of the Polyglot Search Translator a tool for

improving searches in systematic reviews: a multi-centre

randomized crossover trial

• Clark J1, Carter M1, Honeyman D2, Cleo G1, Auld Y3, Booth D4, Condron P5, Dalais C6, Dern S2, Linthwaite B3, May N7, Munn J8, Ramsay L9, Rickett K6, Rutter C10, Sanders S1, Smith A11, Sondergeld P10, Wallin M11, Beller E1

• 1 Centre for Research in Evidence Based Practice, Bond University, Australia

• 2 Bond University Library, Bond University, Australia

• 3 Gold Coast Health Library Service, Gold Coast University Hospital, Australia

• 4 University Library, University of Newcastle, Australia

• 5 University Library, University of Melbourne, Australia

• 6 University Library, University of Queensland, Australia

• 7 Sturt Library, Flinders University, Australia

• 8 Centre for Teaching and Learning, Southern Cross University, Australia

• 9 University Library, Australian Catholic University, Australia

• 10 University Library, Queensland University of Technology, Australia

• 11 Hunter New England Health Libraries, NSW Health

• 12 University Library, Southern Cross University, Australia

Background

• Searching the literature to identify relevant studies is an

important part of a SR

• Translating the search to run in multiple databases is

needed for a good SR

• To improve the speed of this the Polyglot Search

Translator (PST) was developed.

Objectives

• To determine whether a PST-assisted search translation

improves the speed without an increase in errors when

translating search strings from one database to another.

Methods

• Participants (n=20) were recruited, 4 dropped out leaving

16 (10 of whom completed all 10 translations)

• Participants were randomly allocated ten search

strategies from a pool of 20

• They were then randomly assigned to either use the PST

to help translate it, or to not use it

• We compared how long it took, how many errors were

made and how many search results they found

Results

• A PST assisted search strategy translation was faster

with, a mean time saving of 14 minutes and 18 seconds

per translation.

• If you approximate a translation time of 1 hour per search

this meant a 24% reduction in time.

Figure 1 Mean time taken (minutes) to translate searches

45.1

62.6

79.1

66.8

72.8

70.6

44.7

41.5

27.6

33.6

26.8

28.3

23.7

42.0

11.4

30.8

43.8

49.7

23.5

49.8

52.5

56.7

30.9

17.7

22.5

15.5

24.5

18.1

20.0

5.8

0 10 20 30 40 50 60 70 80 90

All databases

Ovid MEDLINE to All databases

Ovid MEDLINE to CINAHL

Ovid MEDLINE to Cochrane

Ovid MEDLINE to Embase

Ovid MEDLINE to Pubmed

Ovid MEDLINE to Scopus

Ovid MEDLINE to Web of Science

PubMed to All databases

PubMed to CINAHL

PubMed to Cochrane

PubMed to Embase

PubMed to MEDLINE

PubMed to Scopus

PubMed to Web of Science

PST-A method

Manual method

Search hits: median percentage difference from the expected number to

be found

PST-A method Manual method

Translation

Median %

DifferenceRange

Median %

DifferenceRange

PubMed-Web of Science -1.47 (-87.3 to 600.0) 22.8 (-77.2 to 625.7)

PubMed-Scopus 768.3 (-95.2 to 18700.0) 46.1 (0.8 to 28000)

PubMed-MEDLINE -12.9 (-66.7 to 82.4) -17.3 (-57.8 to 160.2)

PubMed-Embase -2.5 (-60.7 to 86.6) -17.9 (-37.9 to 19.8)

PubMed-Cochrane -1.1 (-100.0 to 486.9) 34.6 (-48.5 to 1467.2)

PubMed-CINAHL 13.8 (-38.6 to 1262.5) 43.0 (-4.6 to 300.0)

PubMed - All databases -1.5 (-100.0 to 18700) 2.1 (-77.2 to 28000.0)

MEDLINE-Web of Science 11.7 (-75.9 to 596.7) 7.0 (-78.3 to 731.9)

MEDLINE-Scopus 22.5 (-41.2 to 2012.5) 21.9 (-91.4 to 7424.8)

MEDLINE-PubMed 2.4 (-98.8 to 753.1) 20.4 (-97.5 to 4658.5)

MEDLINE-Embase -23.1 (-99.6 to 7.2) -12.4 (-89.7 to 13.7)

MEDLINE-Cochrane2.4 (-100.0 to 39716.7) 1.4 (-98.9 to 1562.9)

MEDLINE-CINAHL 40.6 (-84.9 to 563.0) 104.0 (-36.6 to 2984.7)

MEDLINE - All databases3.0 (-100.0 to 39716.7) 7.0 (-98.9 to 4658.5)

All databases1.7 (-100.0 to 39716.7) 6.3 (-98.9 to 4658.5)

Errors: mean number of total errors

14

15

7

27

13

15

17

2

9

6

8

18

13

9

0

12

14

1

16

17

23

3

5

7

0

7

15

6

2

0 5 10 15 20 25 30

Total

Ovid MEDLINE-total

Ovid MEDLINE-CINAHL

Ovid MEDLINE-Cochrane

Ovid MEDLINE-Embase

Ovid MEDLINE-PubMed

Ovid MEDLINE-Scopus

Ovid MEDLINE-Web of Science

PubMed-total

PubMed-CINAHL

PubMed-Cochrane

PubMed-Embase

PubMed-MEDLINE

PubMed-Scopus

PubMed-Web of Science

Mean total errors Polyglot (n=95) Mean total errors Manual (n=103)

Errors: Number of type of errors

Type of error All databases Polyglot (n=95) All databases Manual (n=103)

Wrong wildcard or truncation syntax - recall 28 21

Missing or added truncation or wildcard - recall 6 14

Missing or added truncation or wildcard - Precision 0 1

Truncated term at wrong point - recall 2 5

Truncated term at wrong point - Precision 0 3

Missing subject term - recall 3 3

Missing keyword term - recall 4 12

Restricting to major subject terms - recall 3 3

Not exploding subject terms - recall 7 8

Exploding subject terms - precision 3 17

Incorrect MeSH conversion - recall 3 5

Incorrect MeSH conversion - Precision 3 1

Incorrect adjacency syntax - recall 5 1

Used incorrect adjacency - recall 0 2

Used incorrect adjacency- Precision 3 6

Incorrect field syntax - recall 9 2

Incorrect field syntax - precision 0 1

Chose incorrect fields - recall 4 13

Chose incorrect fields - precision 7 13

Incorrect phrase translation - recall 6 3

Incorrect phrase translation - Precision 1 6

Extra out of place characters inserted - recall 8 4

Spelling mistake in search - recall 1 7

Incorrect boolean operator - recall 2 1

Incorrect search structure - recall 6 7

Extra components added - recall 0 0

Extra components added - Precision 3 0

Total unique errors 117 159

Conclusion

• For people skilled in conducting searches of the literature

the PST can increase the speed of translating search

strategies h which this can be done, without a decrease in

quality