C O M M E N T A R Y

Evidence-based abstracts: what research summaries shouldcontain to support evidence-based medicinejbr_266 154..158

Reza Assadi MD HSCR MPH,1 Nazila Zarghi MSc,2 Alireza Sapehri Shamloo3 andYasaman Nikooiyan3

1Education Development Center, 2Nursing School and 3School of Medicine, Mashhad University of Medical Sciences, Mashhad, Khorasan,Iran

AbstractThe practice of evidence-based medicine involves the integration of individual clinical expertise with the bestavailable external clinical evidence from the systematic retrieval of the most current literature. Clinicians need toconduct a critical appraisal of the medical articles they access. However, clinicians in developing countries usually lackaccess to the best resources for evidence-based practice (EBP). The abstracts of 100 of the most recently publishedrandomised controlled trials were used in the present study. These abstracts were critically appraised using a newquestionnaire. Questions 1 to 8 were answerable by 38%, 26%, 52%, 23%, 12%, 53%, 36% and 12%, respectively,of the examined summaries. EBP requires better access to medical resources. Therefore, the summaries of relevantstudies should be complete and self-sufficient to support EBP. This means a research summary should adequatelyreport the findings of a clinical trial without needing to access the full text.

Key words: education-professional, epidemiology, evidence-based practice, general practice, researchmethodology.

Introduction

Evidence-based medicine (EBM), a concept first defined bySackett in the late 1990s, refers to the conscientious, explicitand judicious use of the most up-to-date and best availableevidence to help make decisions on the care of individualpatients.1 The practice of EBM involves the integration ofindividual clinical expertise with the best available externalclinical evidence. It requires the systematic retrieval of themost current literature, based on which the best decision fora patient is made with due consideration of his/her values.This approach is widely accepted and is now taught inmedical schools; it is also commonly used for clinicaldecision-making worldwide.2

To date, evidence-based practice (EBP) is not only popularin clinical medicine but also in other aspects of healthcare. Asproof, the Cochrane Collaboration has been established, anda number of researchers continue to conduct systematicreviews on the effects of different healthcare interventions.High-level evidence is provided, and more EBP journals con-tinue to launch.3

The rapid increase in published studies and the continuingemergence of online medical resources are advantageous to

EBM. Online resources provide a wealth of major studies,and many other sites grant access to synthesised resources,such as abstracts, systematic reviews and synopses, whichprovide high levels of evidence and reshape the evidencepyramid.4 Moreover, the rapid development in mobile tech-nology has made access to medical information easier, evenat bedsides, in clinical settings.5

The first step in EBP is the conversion of the need forinformation into clinically accurate and answerable ques-tions commonly expressed as population/problem, inter-vention, comparison and outcome (PICO). The clinicalquestions may be related to aetiology, differential diagnosis,diagnostic tests, prognosis, treatment, prevention or self-improvement.6

The second step is to find the best evidence with which toanswer the questions. The principles are well defined whensearching for published research. Studies are categorisedbased on the methodologies they employed in variouslevels. Whenever a research is located on a higher level onthe pyramid of evidence, such study’s findings are consid-ered more reliable. For example, in the case of therapeuticinterventions, randomised controlled clinical trials that usestronger methods and statistical data are assumed to betrustworthy sources of evidence. The emergence of system-atic reviews, synopses and synthesis resources has modifiedthe pyramid of evidence in recent years. Nevertheless, theseresources serve as excellent and useful materials especially

Correspondence: Dr Reza Assadi, Education Development Center,Mashhad University of Medical Sciences, Daneshgah Ave.,Mashhad 913791-3316, Islamic Republic of Iran. Email:reza_asady@yahoo.com

bs_bs_banner

doi:10.1111/j.1744-1609.2012.00266.x Int J Evid Based Healthc 2012; 10: 154–158

© 2012 The AuthorsInternational Journal of Evidence-Based Healthcare © 2012 The Joanna Briggs Institute

for healthcare professionals who are too busy to follow thesteps of EBM when dealing with new clinical questions.

As soon as clinicians find new evidence from any level orresource, they critically appraise it for validity (i.e. closenessto the truth) and usefulness (i.e., clinical applicability). Thefinal step involves the integration and application of theirappraisal to their clinical expertise. Further, clinicians are alsorecommended to evaluate their clinical performance regu-larly in order to determine if their approach results in betteroutcomes for patients.7

Both the types and numbers of medical resources arerapidly expanding, and some have undergone criticalappraisal by experts working on the literature to producecommercial references for clinical practice. This approach isin accordance with the idea of Smith: ‘Most of the clinicalquestions generated in consultations go unanswered, mostof them can be answered, usually from electronic sources,but it is time consuming and expensive’. Therefore, the idealinformation source should be directly relevant, contain validinformation and is accessible with minimum amount ofwork. Despite the obvious usefulness of these resources,however, they cannot answer all clinical questions. They arecommercially produced, and so access to them requiresregular purchase.8

Searching for literature is now more comprehensive andeasier than ever, but it is not exactly devoid of problems. Forprimary studies, after they are found, their title must reflectthe content, and their results must be valid and useful. Atthis point, a critical appraisal should be done by clinicians. Assoon as the appraisal is completed, clinicians can proceed tothe next step.

This step is access to pre-filtered evidence, which could savepractitioners’ time and provide them opportunity to rely onthe other experts’ findings. It would be possible by systematicreviews whose aim is synthesizing the results of multipleoriginal studies using certain strategies to delimit the bias andsystematic errors. Obviously, they would be more efficient inclinical practice than single research studies, and even ran-domised controlled trials (RCTs). Moreover, if systematicreview is followed by meta-analysis, its quality of evidencewould increase because of calculating effect size in order todetermine the effectiveness of an intervention.

Although systematic reviews are at the top of the evidencepyramid, they are only emerging; on the other hand, lack ofsuch studies for all clinical questions as well as lack of accessto online resources, especially in developing countries, causemany clinicians to focus on other kinds of studies such asRCTs in many clinical situations. Therefore, the present studyaimed to designate a short form for abstract appraisal bywhich the RCTs could be appraised based on their abstractsfor application in daily practice, especially when theirfull-texts are not available.

Several critical appraisal tools have been developed, andsome of the more common ones include JADAD, RAMMboand FRISBE.9–11 These tools are used to evaluate the usabilityand applicability of a published article for EBP, and they areavailable in separate worksheets for various types of studies.They involve questions that assess the quality of the published

article and determine if its findings can be applied to actualpractice. These questions either evaluate the methodologiesused by the study or identify the appropriateness of thesampling method employed for a particular clinical situation.

Although there are many standard appraisal tools andreporting guidelines such as CONSORT for RCTs, they aretoo complicated to apply in clinical settings. On the otherhand, they appraised studies in detail and could not coverthe EBM elements and statistical indexes for clinical practicethat are necessary for everyday clinical situations.

Appraisal tools are dependent on full textsThe major pre-requisite of EBM is access to online resources,without which all training is useless. Therefore, a relevantquestion to ask is whether clinicians in developing countrieshave ready access to the best current resources for EBP. Ourliterature review confirms the hypothesis that clinicians indeveloping countries have limited access to the bestresources for EBP.12

Moreover despite international attempts for provision full-text for developing countries – for example, the HINARIagreement aims to provide low cost or even open access toresearch studies and major online biomedical and relatedsocial sciences journals – the literature reveals that thesearrangements are neither available everywhere nor haveenough coverage.13

Evidence-based abstracts (EBAs)The present study aims to evaluate the most commonly usedcritical appraisal tools. The use of these tools on researchabstracts is hypothesised to be a possible alternative toaccess to full texts.

Our hypothesis is motivated by two facts. First, mostresearch journals are funded by commercial publishers andoffer individual and institutional subscriptions for access tothe full texts of articles. Second, in many developing coun-tries, access to full texts is only available to academic insti-tutions. This means that private clinicians have to adopt oneof the following approaches:1 Purchase online resources individually on demand.2 Retrieve evidence from free full-text, open-access

resources.3 Retrieve evidence from free abstract resources.

Purchase online resources individually on demandClinicians working in the private sector can use pay-per-viewservices to access research manuscripts at costs ranging fromUS$15 to US$50 per paper. However, the average incomeof clinicians in developing regions is much lower than thatof their colleagues (even up to 24 times) in developedregions.14 As a further illustration, visiting a general practi-tioner (GP) in a private office in Iran costs about US$6.Therefore, practicing EBP for GPs in developing countriesremains to be quite expensive at present.15

Retrieve evidence from free full-text, open-accessresourcesAn evaluation of PubMed showed that in light of the emer-gence of open-access, full-text literature, online searches

Evidence-based abstracts 155

© 2012 The AuthorsInternational Journal of Evidence-Based Healthcare © 2012 The Joanna Briggs Institute

return a large number of results on published studies. To date,many subscription-based journals offer the option of openaccess to authors for a fee. Whereas some journals, such as theWHO Bulletin, do not charge authors and are funded byinternational organisations, most open-access journalsrequire authors to pay a publication fee, which can vary fromas small as US$100 to more than US$1500. Open access is notonly advantageous to journals but more so for authors whocan have their manuscripts viewed and cited more often.

These same resources, however, are not appropriate forclinicians in developing countries for two reasons. First, open-access journals tend to publish manuscripts from researchersin developed countries more regularly. Therefore, accessingthese journals may not be appropriate for clinicians in devel-oping countries as the health problems they deal with largelyvary from those handled by their counterparts in developedcountries. Second, an evaluation of the proportion of open-access manuscripts within all publications in PubMed revealsthat only 18% of MEDLINE papers are freely available infull-text versions. In other words, a large number of studiesare not yet available for free.

Retrieve evidence from free abstract resourcesSome databases provide free access to the abstracts ofpapers. Therefore, a good approach is to find researchabstracts from resources that provide free access, such asPubMed. This approach is the only possible and cost-effective solution for clinicians in developing countriesattempting to practice EBP. However, the applicability of thisapproach still needs to be verified further as few studies ofthis kind were found in the current research. These textswere also examined to assess if they were accurately pre-pared by the authors in a manner that can promote EBP.16

Methods

The abstracts of 100 of the most recent published RCTs wereexamined to assess the usefulness of these texts. They wereretrieved from MEDLINE using PubMed in March 2011, with‘randomised controlled trial’ as the keyword; the data werethen sorted by publication date using the ‘display setting’option of the web site. Various worksheets available fordifferent types of studies were used, and RCTs were selectedas these are of great value to clinicians.

Development of the checklistAll available tools for RCTs appraisal such as CONSORT,FRISBE, CASP and JADAD, as well as EBM essential indexesfor clinical application were reviewed and considered in newchecklist. Then, its validity was confirmed by a medicalresearch methodologist and its reliability was confirmed byinter-rater reliability between reviewers. Some criteria weredescribed objectively for each question in order to facilitatethe reviewing process.

More than 300 000 records were retrieved, and the first100 were included in the present study for critical appraisal.Studies without abstracts, those which used methodologiesother than RCTs, and those that involved animal or in vitro

trials were excluded. Using the three most commonly usedcritical appraisal worksheets, namely, FRISBE, JADAD andRAMMbo, eight questions that met the criteria were used toevaluate the abstracts critically (Table 1). The abstracts wereevaluated by ‘Yes’, ‘No’ or ‘Not Clear’ replies to the eightquestions. Data were initially collected in Microsoft Excelversion 2007 and then were inputted to Social PackageStatistics Software (version 11.5) for further analysis.

Critical appraisals were conducted by two researchersunaware of each other’s work, and then their output wasinterpreted by a third researcher. Whenever the first tworesearchers were uncertain about the assessments or hadconflicting opinions, the case was evaluated by a thirdresearcher who was a lecturer and author of a book on EBM.

Results

The critical appraisals of 100 abstracts with eight questionsand options of ‘Yes’, ‘No’ or ‘Not Clear’ were conducted bytwo researchers. Discussions were made with a thirdresearcher whenever there were conflicting opinions orunclear responses to a query (Table 2).

Question 1: Was the assignment of patients to treat-ments randomised?Randomisation is a key step in generalising the findings of astudy in that it confirms whether the observed differences

Table 1 Questions extracted from three critical appraisal forms andused for the study

No. Question Description of criteria forappraisal

Q1 Was the assignment ofpatients to treatmentsrandomised?

• Trial design• Sample size• Number of each group

Q2 Were the groups similar atthe start of the trial?

• Eligibility criteria• Matching groups

Q3 Aside from the allocatedtreatment, were groupstreated equally?

• Intervention intended foreach group (Similar?)

• Bias limitations?Q4 Were all patients who entered

the trial accounted for?Were they analysed in thegroups to which they wererandomised?

• Allocation concealment• Random allocation

Q5 Were measurements objectiveor were the patients andclinicians kept “blind” tothe treatments beingreceived?

• Blinding (ways of blinding?e.g. single, double

Q6 How large was the treatmenteffect?

• Number of analysed ineach group

• CI?, P-values?Q7 How precise was the estimate

of the treatment effect?• indexes for treatment effect

(RR, ARR, NNT)• number of each group

Q8 Was proper information forIntention-to-Treat Analysisavailable?

• Loss to follow up• Dropout rate• ITT

156 R Assadi et al.

© 2012 The AuthorsInternational Journal of Evidence-Based Healthcare © 2012 The Joanna Briggs Institute

among interventions and the control group are not relatedto a selection bias. In the present appraisal, 38% of theselected articles appropriately presented the randomisation.This is a major consideration for evidence-based practitio-ners when assessing if the findings of a study are applicableto actual practice.

Question 2: Were the groups similar at the start ofthe trial?A statement on the matching of groups is essential. This willassure practitioners on the effectiveness of the interventiondone in the experimental group versus the control group.However, in our investigation, less than one-third of theexamined abstracts met this criterion.

Question 3: Aside from the allocated treatment,were the groups equally treated?The groups involved in the study should also be equallytreated, although the experimental group is more vulnerableto unintentional overtreatment and more attention by theresearchers. Unfortunately, our results showed that only halfof the examined abstracts mentioned this issue.

Question 4: Were all patients who participated inthe trial accounted for? Were they analysed in thegroups to which they were randomised?A statement on the appropriate selection of the samples forthe study is necessary, and their proper allocation to theright group should be clearly stated. In the present work,only 23% of the selected summaries addressed thisrequirement.

Question 5: Were the measurements objective, orwere the patients and clinicians kept ‘blind’ to thetreatments received?‘Blindness’ is important, and its level (single, double or triple)should be indicated by the authors. This will clarify withpractitioners how far the interventions are from observer andmeasurement biases. The minimum score was related to thisquestion with 12%, for reporting blindness.

Question 6: How large was the treatment effect?The P values and confidence interval should also be stated.This will inform practitioners about the size of the trial. Pvalues larger than 0.05 or less than 0.001 are not recom-

mended; exact numbers should be used. In the presentinvestigation, only half of the studies examined used therecommended values.

Question 7: How precise was the estimate of thetreatment effect?In RCTs, the risk rates are calculated in order to measure theeffects of the experimental treatment compared with routineor the absence of treatment. In this regard, five values shouldbe indicated: experimental event rate, control event rate,outcome (i.e., the desired clinical improvement), rate of lackof outcome in all groups (A, B, C and D; Table 3), and totalnumber of subjects. Around one third of abstracts couldacquire the score of treatment effect estimation.

Question 8: Intention-to-Treat AnalysisIn almost all clinical trials, there will always be subjects whowill drop out from the intervention and control groupsdepending on the intervention side effects, death, or otherreasons. The number of subjects who dropped out from eachgroup, particularly those from the experimental group, isimportant in determining the effectiveness of the interven-tion. Therefore, a good summary should state the drop-outrates for both groups. In the current investigation, 18% ofthe abstracts mentioned their drop-out rates. This makesintention-to-treat analysis impossible for practitioners at-tempting to practice EBM according to literature summaries.

Discussion

EBM is an excellent technique for clinical decision making interms of determining which one of its three main parts isusing best as evidence.6 Apparently, the full text of articleshould be used for this approach. However, most clinicians,especially those in developing countries, do not have accessto these resources. Therefore, in the present study, theusability of abstracts for EBP was assessed. Whether theseabstracts were appropriately written and whether they canbe used by clinicians who do not have access to full-textresources were determined.

EBAsEBP requires access to medical resources, but whenever thisis not feasible as is the case in many regions worldwide,14,17

some other strategies should be used. One option is the useof research abstracts that are often available from free onlineresources, such as PubMed. The current research examinedthis issue, but the findings showed that the current style ofabstract writing in medical journals is not applicable tosupport EBP, and access to full texts is the only solution forEBP.

Table 2 Frequency of appraisal questions in one hundred of mostcurrent published RCTs

Yes No Not clear

Question one 38 54 8Question two 26 62 12Question three 52 30 18Question four 23 70 7Question five 12 87 1Question six 53 46 1Question seven 36 64 0Question eight 12 56 32

Table 3 Some rates should be clearly stated in research abstract

Outcome Not Outcome

Experimental group A BControl group C D

Evidence-based abstracts 157

© 2012 The AuthorsInternational Journal of Evidence-Based Healthcare © 2012 The Joanna Briggs Institute

The questions in three appraisal tools (JADAD, RAMMboand FRISBE) were evaluated, and based on current practicesin the presentation of research abstracts, a guide for abstractpreparation when reporting the findings of RCTs was pre-pared. This approach helps make EBP possible for any clini-cian who has access to the Internet and who knows how touse PubMed or other free online resources, such as ‘Googlescholar’, to find research abstracts. EBA is a researchsummary report that makes EBP possible for clinicians(Table 4).

However, the use of EBAs involves a major limitation aswell. The procedural and technical issues in clinical interven-

tions and the details found in figures and tables cannot beadequately presented in EBAs. Nevertheless, EBAs can stillassist clinicians in better selecting articles to be bought orsearched.

Conclusion

EBAs help improve the EBP approach for clinicians andhealthcare providers who do not have access to full-textresources. However, the collaboration of the editorial boardsof medical journals is needed. They need to modify theirguidelines for abstract presentation so that these abstractscan provide more adequate and relevant information tosupport EBP. Better access to medical resources will result inbetter care for patients, especially those in developingregions and, overall, the improvement of global health in thelong run.

References1. Staunton M. Evidence-based radiology: steps 1 and 2 – asking

answerable questions and searching for evidence. Radiology2007; 242: 23–31.

2. Oude Rengerink K, Thangaratinam S, Barnfield G et al. How canwe teach EBM in clinical practice? An analysis of barriers toimplementation of on-the-job EBM teaching and learning. MedTeach 2011; 33: e125–30.

3. White PJ. Evidence-based medicine for consumers: a role forthe Cochrane Collaboration. J Med Libr Assoc 2002; 90: 218–22.

4. Hooper-Lane C, Combs AM, Feldstein D. Finding the best avail-able evidence: what’s new? WMJ 2005; 104: 18–21.

5. Fontelo P. Consensus abstracts for evidence-based medicine.Evid Based Med 2011; 16: 36–8.

6. Sackett DL, Rosenberg WM. The need for evidence-based medi-cine. J R Soc Med 1995; 88: 620–4.

7. Carter MJ. Evidence-based medicine: an overview of key con-cepts. Ostomy Wound Manage 2010; 56: 68–85.

8. Smith R. What clinical information do doctors need? BMJ 1996;313: 1062–8.

9. Clark HD, Wells GA, Huët C, McAlister FA, Salmi LR, FergussonD. Assessing the quality of randomized trials: reliability of theJADAD scale. Control Clin Trials 1999; 20: 448–52.

10. The Royal Australian College of General Practitioners. GP Formsand Guides. Accessed April 2012. Available from: http://www.racgp.org.au/news/22462

11. Duke University. Expanded critical appraisal worksheet with keylearning points. Accessed April 2012. Available from: http://www.mclibrary.duke.edu/subject/ebm/FRISBE.pdf

12. Sara Gwynn S, Tariq AS. Access to research information indeveloping countries. Lancet 2011; 377: 807.

13. Koehlmoos TP, Smith R. Big publishers cut access to journals inpoor countries. Lancet 2011; 377: 273–6.

14. Smith H, Bukirwa H, Mukasa O et al. Access to electronic healthknowledge in five countries in Africa: a descriptive study. BMCHealth Serv Res 2007; 7: 72.

15. Shalileh K, Mahdanian A. Family physicians’ satisfaction in Iran:a long path ahead. Lancet 2010; 376: 515.

16. Dawes M, Pluye P, Shea L, Grad R, Greenberg A, Nie JY. Theidentification of clinically important elements within medicaljournal abstracts: Patient-Population-Problem, Exposure-Intervention, Comparison, Outcome, Duration and Results(PECODR). Inform Prim Care 2007; 15: 9–16.

17. Geyoushi BE, Matthews Z, Stones RW. Pathways to evidence-based reproductive healthcare in developing countries. BJOG2003; 110: 500–7.

Table 4 What each section of an evidence-based abstract shouldcontain

Section Content

Introduction/backgroundand study objectives

Expression of the main objective ofthe study in some detail to guidepractitioners for understanding theapplicability of current interventionfor their own clinical question

Material and Methods/Design, Setting orParticipants

Randomization technique used forselection of the samples andallocation of them should beinformed. Scope of blindness of thestudy, and whoever included,patients, clinicians, researchers(single, double or triple blind)should be stated.

Similarity of the patients andmentioning the majorcharacteristics of them (it will helppractitioners to understand if theycan assume current sample thesame to their own)

Result Being control and experimentalsamples the same and the maincharacteristic of this similarityshould be informed.

The most important outcomes of thestudy should be written with Pvalue and CI.

The seven golden numbers:, numberof samples in control andexperimental groups, totalparticipants, drop-out rates in both,control and experimental eventrates. These rates should be writtenin format of mean � SD. Theserates help practitioners to measurerisks, effect size and power of theintervention)

Discussion Similarity of treatment, care andfollow up should be stated and lackof overtreatment between groupsto be clarified.

Conclusion Any clue about value of the results orcomment for practitioners guidingthem in applying outcomes forown.

158 R Assadi et al.

© 2012 The AuthorsInternational Journal of Evidence-Based Healthcare © 2012 The Joanna Briggs Institute