The Journal Club: Exploring New

Avenues

Dr. Imad Salah Ahmed Hassan MD FACP FRCPI MemAcadMEd MSc MBBSChairman, Knowledge Translation Committee

Department of Medicine, KAMC,KSA

The Journal Club: One of EBM

Implementation Tools

Objectives Definition History Goals Benefits to practice site How to select and present an article What next?

Objectives (cont’d) Journal club questions Conducting a journal club

The presentationLearning OutcomesLearning/Teaching MethodsAssessment

Factors associated with successful journal clubs

Journal club models

Definition

A group of individuals who meet regularly to critically discuss applicability of current articles found in medical journals.

Kleinpell RM. Rediscovering the value of the journal club. Am J Crit Care. 2002;11:412-4.

At the beginning…• The first journal club was founded

by Sir James Paget, a British Surgeon in the mid-1800’s• “Small room over a baker’s shop near the Hospital-gate

where we could sit and read the journals”

At the beginning…• Sir William Osler established the first formalized

journal club at McGill University in Montreal in 1875.

• Purpose: “for the purchase and distribution of periodicals to which he could ill afford to subscribe”

Medical Knowledge• Update Knowledge and support life-long learning. Skills: Training in evidence based medicine skills• JCs enhances literature searching skills, critical

appraisal etc.

Understand scientific concepts in Epidemiology and Biostatistics:• Basic study designs• Basic statistical approaches• Basic outcomes assessment methods.

Objectives of a Journal Club today?

Using the Journal Club to teach and assess competence in practice-based learning and improvement: a literature review and recommendation for implementation. Surv Ophthalmol. 2005 Nov-Dec;50(6):542-8.

Communication Skills, Socialization, Democracy (Attitude)• Develop staff skills in presentation. Change of Practice (Quality Improvement)• Promotes evidence-based practice/clinical

care.

Enhances Staff’s Academic Output

• Promotes Research and Audit.• Promotes Faculty Publications.

Objectives of a Journal Club today?

Using the Journal Club to teach and assess competence in practice-based learning and improvement: a literature review and recommendation for implementation. Surv Ophthalmol. 2005 Nov-Dec;50(6):542-8.

•Letter to the editor.1• CAT Journals.2• CAT and JC web

publications.3

Enhances Staff’s Academic Output

Knowledge Translation: Evidence to Practice

Journal clubs have been advocated as a bridge between Research and Practice.

Goodfellow LM. Can a journal club bridge the gap between research and practice? Nurse Educ. 2004 May-Jun;29(3):107-10.

The Journal Club: Does it work?BMC Med Educ. 2006; 6: 43.Do community medicine residency trainees learn through journal club? An experience from a developing country?Saima Akhund et al Human Development Programme, Aga Khan University, Karachi, Pakistan

AbstractMethods• Journal club effectiveness criteria were searched using

electronic search databases. • Ninety percent of residents and alumni of Community

Medicine Residency Programme participated voluntarily in a confidential survey.

The Journal Club: Does it work?

ResultsThe CMR-JC was regularly conducted.

More than 95% of residents attended. (Total residents in the CMR-Programme: 32).

Twenty-seven out of 29 current residents/alumni responded to the anonymous questionnaire.

Acquisition of critical appraisal skills (23 respondents) and keeping up with current literature (18 respondents) were the two most important objectives achieved.

Respondents recommended improved faculty participation and incorporating a structured checklist for article review.

Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized controlled trial. Linzer M et al

To determine if a journal club improves house-staff reading habits, knowledge of epidemiology and biostatistics, and critical appraisal skills.

Randomized 44 medical interns to receive either a journal

club or a control seminar series.

A test instrument developed by the Delphi method was administered before and after the interventions (mean, five journal club sessions).

The Journal Club: Does it work?

•By self-report, 86% of the house staff in the journal club group improved their reading habits vs 0% in the control group. 1• Knowledge scores increased more in the journal club group

than in the control group, and a trend was found toward more knowledge gained as more sessions were attended. 2

• Ability to appraise critically a test article increased slightly in each group, but there was no significant difference between the groups. 3

The Journal Club: Does it work?: A Randomized Study.

Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized controlled trial. Linzer M et al

We conclude that a journal club is a powerful motivator of critical house-staff reading behaviour and can help teach epidemiology and biostatistics to physicians-in-training.

Conclusion

Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized controlled trial. Linzer M et al

A literature search was undertaken using 10 databases was made as to whether studies have found journal clubs for physicians in training to be:

1. Effective for improving patient care 2. Teaching critical appraisal skills 3. Improving reading habits4. Increasing knowledge of clinical epidemiology and

biostatistics5. Increasing the use of medical literature in clinical practice.

The Journal Club: Does it work?

Ebbert JO et al. The journal club in postgraduate medical education: a systematic review. Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, USA. Med Teach. 2001 Sep;23(5):455-461

• It is concluded that journal clubs may improve knowledge of clinical epidemiology and biostatistics, reading habits, and the use of medical literature in practice.

• A multi-centre, randomized controlled trial of journal clubs is needed to assess whether journal clubs improve critical appraisal skills.

Conclusion

Impact of a medical journal club on house-staff reading habits, knowledge, and critical appraisal skills. A randomized controlled trial. Linzer M et al

Ebbert JO et al. The journal club in postgraduate medical education: a systematic review. Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, USA. Med Teach. 2001 Sep;23(5):455-461

Characteristics of Successful Journal Clubs

One• Regular and anticipated meetings.

Two• Mandatory presentation and attendance.

Three• Clear short- and long-term purpose: Education,

Publications, Quality, Research etc .

Four• Appropriate timing of meeting.

Five• Incentives: Material, Food, Prizes etc.

Six• A trained journal club leader to choose

papers and lead discussion.

Seven

• Circulating papers prior to the meeting.

Eight• Using the internet for wider dissemination

and data storage.

Nine

• Using established critical appraisal processes (EBM Staff Training).

Ten

• Summarizing journal club findings.

Clinical Query: Foreground Question

Clinical Query: Foreground Question

Structure of JC Clinical Query: Foreground Question PICO Article Selection: Searching for Evidence/Literature Search Appraising the Evidence: Critical Appraisal Presentation Critique and summary Recommendations:

New research Change or audit of current practice Writing a letter to the editor Publishing your appraisal in a CAT journal or website

(own or in the WWW)

Clinical Query: EBM Approach

Ask clinical questions

Acquire the best evidence

Appraise the evidence

Applyevidence to Your patient

5A’s !!Assess

effectiveness, efficiency of

EBM process

Evidence-based Practice

Ask clinical questions

PICO

Ask: Refine ability to convert need for information (e.g., therapy, prevention, diagnosis, prognosis) into an answerable question.

PICO format for foreground questions:Educational Prescription

Ask Clinical Questions: PICO

Patient/Population OutcomeIntervention/

Exposure Comparison

Components of Clinical Questions

In patients withacute MI

In post-menopausal

women

In women withsuspected

coronary disease

does early treat-ment with a statin

what is the accuracy of

exercise ECHO

does hormonereplacement

therapy

compared to placebo

compared to exercise

ECG

compared to noHRT

decrease cardio-vascular mortality?

for diagnosingsignificant

CAD?

increase therisk of

breast cancer?

Article Selection

Appraisal

Presentation

Difficulties in the JC

Article Selection

Select articles from peer reviewed, quality journals. Try to select recent articles ( write a Letter to the

Editor!). Choose a research paper rather than a review article

that will not have a Methods section. Use the McMaster MORE tool to choose your article.

http://hiru.mcmaster.ca/MORE/InclusionCriteria.htm A pair of articles may be good for a lively discussion

particularly if they have opposite conclusions.

Literature Search and Retrieval of Article(s)

“Pubmed”: www.pubmed.org Limits Clinical Query PubmedviaPICO AskmedlineJournal Websites: www.acpjc.org www.hematology.org/education/training/index.cfm www.cancer.gov/clinicaltrials

McMaster Online Rating of Evidence (MORE)

McMaster University has created the McMaster Online Rating of Evidence (MORE) system to identify the highest-quality published research.

The ACP Journal Club uses the MORE system to select their articles.

Specific inclusion criteria have been delineated in order to distinguish papers with the highest scientific merit.

McMaster Online Rating of Evidence (MORE)

Articles that have passed this screening are then rated by clinicians on their clinical relevance and newsworthiness, using a graded scale.

With the help of your mentors and colleagues, you can use these criteria and the rating scale as informal guidelines to ensure that your chosen article merits presentation.

Critical Appraisal

You should attempt to critically appraise the article with help from your mentor.

Use web-based/online CAT sites and Calculators

Critical Appraisal Checklists http://www.gla.ac.uk/departments/generalpracticeprimarycare/ebp/

checklists/#d.en.19536

§         Checklist for an article on treatment or prevention §         Checklist for an article on qualitative research §         Checklist for an article on decision analysis §         Checklist for an article on an educational intervention §         Checklist for an article on prognosis §         Checklist for an article on harm or causation §         Checklist for an article on guidelines §         Checklist for an article on diagnosis or screening §         Checklist for a systematic review §         Checklist for economic evaluations

Critical Appraisal Calculators

Oxford Centre for EBM: http://www.cebm.net/?o=1023

Critical Appraisal: Shortcuts to Enriching your Journal Club

Check for an “Editorial Comment” (usually present in the same journal issue where the article was published).

Check for any “Letters to the Editor” relevant to the article of your choice in PubMed.

Check whether an appraisal of the article has been published in one of the ACP Journal Club issues (https://www.acponline.org/clinical-information/journals-publications/acp-journal-club).

Critical Appraisal: Shortcuts to Enriching your Journal Club

Check whether an appraisal of the article has been published in the wiki Journal Club (https://www.wikijournalclub.org/wiki/Main_Page).

Check whether an appraisal of the article has been published in: Google. Pubmed On-line websites e.g. Slide Share

Use keywords from the article title plus appraisal or journal club in your search terms.

Preparing the Presentation

Prepare your presentation on PowerPoint and discuss it with your mentor.

To make it easy for the trainees the Department of Medicine has created readymade Word Doc and PowerPoint Templates with Hyperlinks.

Templates 1.Doc 2.Ppt

How to Use the Template1. Go through the template after selecting your JC article and before

you read it!!!2. Try to use McMaster rating before you appraise the article.3. While reading it highlight what you need to include in your

presentation: in a soft copy –use the copy and paste actions for text, tables, graphs etc!

4. Hyperlinks at the bottom are useful for understanding the concepts: critical appraisal, definitions and use of EBM calculators etc

5. Your maximum number of slides is the time allocated for your presentation in minutes provided you stick to the 6X6 slide rule and have rehearsed and timed yourself before-hand!!!!

6. Make sure your supervisor has seen and endorsed what you prepared!

Slide 1

• EBM Journal Club number:

• Date of meeting:• Name of Presenter:• Name of Supervisor:• Name of JC Facilitator/Time

Keeper:

Slide 2

• Source (Name of Journal):• Title of Article• Type of Question: Diagnostic

Therapeutic Prognostic Economic Other:

• Type of Study: RCT Retrospective Cohort Cross-sectional Case-Control Other:

• Date published:• Number of Pages:• Quality of journal i.e. impact factor

(get from Google!)

Slide 3

• Clinical scenario. Why have we asked this question? PICO

• Describe your search method:• Search Terms/Keywords/Mesh

terms• Search site/Search Engine• Limits used e.g. year of

publication, type of study etc• Or Article of Interest chosen

Slide 4• Paste Abstract here: use several

slides if it is long-remember the 6X6 rule in power-point presentations!

Slide 5

• Relevance• Does it describe a common problem

in your practice?• Does it address an important

outcome ?• Is it Patient or Disease oriented

(DOE vs POEM).• Would it lead to a change in practice

if the study is valid?

Slide 6

• Authors: • Do the authors have a reputation

for well conducted research? (Check their research out-put in Pubmed)

• Possible conflict of interest ? Was it disclosed?

Slide 7

• Introduction Section• Was key literature reviewed ?• Were weaknesses in current

knowledge summarized ?• Is the problem for which research

was done stated adequately?• Are objectives clearly stated ?• Are there definitions of all

important terms ? List them• What is the null hypothesis – if

any ?

Slide 8

• Method Section: Patients • Where Patients described fully in

journal article-Inclusion/Exclusion: Yes/No

• Was the sample chosen representative of the target population ? Explain

• Were the time elements of the study clearly described ? Explain

• What type of bias is present ? Describe them

• What were the confounding variables: Demographics, Co-morbidity, Disease Severity etc?

• Have ethical considerations been addressed ?

Slide 9

• Study Conduct• Intervention (s) used in journal

article:• What are they?• Where they appropriate?• Flow chart of the study

Slide 10

• Study Conduct• Statistical tests of data described

in journal article:• Their meaning• Appropriateness?

Slide 9

• Study Conduct• Outcomes

• Were they clearly described? Y/N

• What are they:• Primary• Secondary

Slide 10

• Results• Were the results presented clearly ?• Were the graphs & tables easy to

understand ?• Are the data overanalyzed ? Explain • Were the results statistically significant

?• Intention to Treat Analysis in Therapy

studies?• Where patient-oriented outcomes

emphasized over disease-oriented outcomes?

• Magnitude of the Results: NNT/NNH in therapy articles? LR+/LR- in diagnostic articles etc?

• Precision of Results: Wide or narrow Confidence Intervals?

Slide 10

• Discussion• Are all of the discussions relevant

to the findings ? • Were some findings excessively

discussed compared to others ? Explain

• Are the results of prior studies presented and insightfully discussed ?

• Does the discussion proceed logically based on data presented ? Explain

Slide 11

• Conclusion• Title: Is it informative? Is it

appropriate? Is it misleading or confusing?

• Is the original research objective directly answered ?

• Do the findings show cause & effect relationship ? Or association relationship ? Or no relationship ?

• In your opinion is this finding justified ? Explain

• Is there evidence that the authors are aware of weaknesses/biases in their study (design or execution) and have clearly stated them? Explain

• Editorial commentary?

Actions after a Journal Club?

Slide 12

One• Are there any changes you

suggest that may improve the study ? List them if available

Two

• Describe your action plan and how you will implement change in practice systems in a reliable way. (Knowledge Translation Exercise)

Three

• New Research: Is there a need to do more research of the subject? Y□ N □ Suggested research if the answer is yes:

Four• Audit: Is there a need to audit

your practice? Y□ N □ Suggest an audit cycle.

Five

• Publication: Letter to the Publisher? Is it worth publishing in EBM CAT journal or website? May be start your own JC website?

Six• Keep a copy of your

presentation in your Personal Development file.

Seven• Establish a Departmental JC

CAT Website..

Slide 13

• Conclusions• Restate the authors' take-home

message followed by your own interpretation of the study:• PICO• Findings• Conclusions with Magnitude• Applicability? Trustworthiness?

ApplicabilitySystem

Preparedness: Availability,

Cost etc

Staff Preparedness: Knowledge,

Skill etc

Patient Factors &

Acceptability:Similar to

study population, Preferences, Values etc

Slide 14• Open the floor for

Discussion

McMaster Online Rating of Evidence MORE

hiru.mcmaster.ca/more/InclusionCriteria.htmhiru.mcmaster.ca/more/RatingFormSample.htm

Useful Links

Critical Appraisal Checklists & Calculatorshttp://www.gla.ac.uk/departments/generalpracticeprimarycare/ebp/checklists/#d.en.19536 Checklist for an article on treatment or prevention Checklist for an article on qualitative research Checklist for an article on decision analysis Checklist for an article on an educational intervention Checklist for an article on prognosis Checklist for an article on harm or causation Checklist for an article on guidelines Checklist for an article on diagnosis or screening Checklist for a systematic review Checklist for economic evaluationsEBM toolbox/CATmaker http://www.cebm.net/?o=1023

Useful Links

Publication siteshttp://www.bestbets.org/background/bets-and-cats.phpJournal of Family Practice POEMs (Patient Oriented Evidence that Matters) http://jfp.msu.edu/http://www.emjournalclub.com/www.cop.ufl.edu/doty/pep/buffingtonffw2008.pp

Useful Links

Acknowledgement

Dr Kanaan Alshammari, MBChB Otago 08’ Senior Internal Medicine Resident,

Dr Ali Al Shehri, SBIM, Assistant Consultant.

KAMC Internal Medicine Residents.

Thanks and Good Luck