Review of Year 1 Biostatistics Epidemiology

• Course occurs in the spring term of Year 1

• Course Director – Judy Rees, BM, BCh, MPH, PhD

• Course has 29 curricular hours

Course Objectives – Content Review

There are 13 course objectives that fulfill Geisel competencies as follows:

• 13 address specific knowledge in the preclinical domain

• 1 addresses clinical skills• 3 address communication skills• 1 addresses components of professionalism• 13 address personal improvement• 6 address health care systems

Biostatistics Epidemiology Course Objectives

Course Objectives – Content Review

• Course objectives are provided in the syllabus on the first page and seem appropriate for the course

• Course objectives are written in the correct format

Objectives: Step I Brochure

• Content of the course objectives correlates well with the content in the Step I Brochure for the category “Quantitative methods”

Objectives: National Organization• The course directors are not aware of a national

discipline-oriented organization in their field that has published course objectives, thus no comparison was made

Session Objectives• Session objectives are provided in the course materials for

each lecture and its corresponding small group, however these session objectives do not match the session objectives on Ilios and some do not use measurable verbs

• The course director clarified that the session objectives in the course materials were the old set, and the objectives in the Ilios system were the correct objectives

• The session objectives on Ilios are written in the correct format using measurable verbs; some use higher level verbs such as contrast, estimate, interpret, and apply

• All session objectives map back to one or more course objectives

Redundancy with other courses• Significant redundancy of course content with other

courses was not found in this course• Searches in Ilios for terms such as “population

statistics” and “epidemiologic research” did not reveal any overlap

• Searches for terms such as “diagnostic testing” and “confounding” revealed that these concepts were introduced in this course, then appropriately expanded upon in Y2 (e.g. “Describe the appropriate use of diagnostic testing in patients with disorders of the reproductive system”; “Explain the concept of confounding variables in smoking and lung cancer”)

Summary regarding Objectives

• Course objectives are well-written and fulfill components of all six Geisel competencies

• Two sets of session objectives exist – one in the course materials and one on Ilios; the most current set of session objectives (on Ilios) is well-written

• All session objectives map back to at least one course objective

• Unplanned and unnecessary redundancy was not found associated with this course

Course Learning Opportunities

• Lecture 15 hrs. (52%)• Small group conferences 12 hrs. (41%)• Review session for final exam 2 hrs. (7%)

Course Learning Opportunities

Innovative pedagogy for small groups• Course offers two formats for conferences: a standard

option where students and faculty work together to solve problems during the conferences, and a flipped option where students do the problems before coming to conferences and then they are discussed

• Students choose the format they prefer for the small group at the start of the course

• Attendance is required for small groups, and students are encouraged to participate (i.e. some students are asked to project their work for the group to view)

Summary regarding Pedagogy

• The percentage of traditional lectures in the course is appropriate

• Small group conferences offer opportunities to apply the material

• Two methods of pedagogy are offered for conferences allowing students to choose the method that best suits their learning style

Assessment

• Two written Quizzes - 40% of course grade• Final Exam - 60% of course grade• Conferences do not count towards the course

grade, however attendance/participation in conferences can help a weak student, i.e. syllabus states: “If a student’s final grade is borderline pass-fail, good participation in small groups may benefit the student.”

Assessment – Quizzes and Exams

• Quizzes consist of approximately 15-20 multiple choice questions; the final exam consisted of 30 multiple choice/short answer questions

• All questions were written in correct formats• Some questions required students to apply their

knowledge (statistical calculations)• Questions correlate well with the session

objectives; all questions were linked to at least one objective

Assessment - Conferences

• Currently conference attendance and participation is not assessed

• Students do not receive feedback regarding their conference performance, however this may not “fit” with the activities for the small groups in the course (i.e. students are solving problems and answers are discussed – they are not doing activities such as physical exam where performance can be critiqued)

Summary regarding Assessment• There are no major problems with the assessments

in the course – questions are written in correct format and they correlate well with the session objectives

• Conferences provide opportunities for students to apply their knowledge and discuss the material further with a faculty facilitator

Measures of

Quality – AAMC

GQ

BASIC SCIENCESGeisel

mean 2008Geisel

mean 2009Geisel

mean 2010Geisel

mean 2011Geisel

mean 2012All schools

means 2012

Behavioral Science 3.1 3.3 3.2 3.3 3.3 3.1Biochemistry 2.7 2.8 2.7 2.5 2.6 2.6Biostatistics/Epidemiology 2.7 2.8 2.9 3.2 3.2 2.8Genetics 2.9 2.8 2.9 2.8 2.8 2.8Gross anatomy/Embryology 3.4 3.5 3.6 3.5 3.6 3.4Histology 2.8 3.0 2.8 2.9 3.1 2.9Immunology 2.8 3.0 2.9 3.0 3.1 3.1Microbiology 3.1 3.4 3.1 3.2 3.3 3.1Neuroscience 3.2 3.2 3.2 3.0 3.0 3.2On Doctoring 3.3 3.3 3.5 3.4 3.5 3.4Pathology 3.1 3.1 3.2 3.1 3.4 3.3Pathophysiology of Disease 3.6 3.7 3.5 3.5 3.5 3.5Pharmacology 3.1 3.5 3.4 3.1 3.1 3.0Physiology 3.5 3.5 3.6 3.6 3.5 3.4

Measures of Quality – AAMC GQ“Indicate how well you think that instruction in Biostatistics/Epidemiology prepared you for clinical clerkships and electives.” [1=poor; 2=fair; 3=good; 4=excellent]

Measures of Quality – Step I

2009* 2010* 2011* 2012* Means 09-12

TRADITIONAL CORE DISCIPLINES

Biochemistry 0.50 0.30 0.40 0.30 0.38

Biostatistics/Epidemiology 0.65 0.90 0.73 0.43 0.68

Genetics 0.40 0.30 0.48 0.28 0.37

Gross anatomy/Embryology 0.50 0.35 0.53 0.33 0.43

Histology/Cell Biology 0.53 0.30 0.40 0.37 0.40

Microbiology/Immunology 0.63 0.42 0.41 0.31 0.44

Pathology 0.55 0.35 0.42 0.26 0.40

Pharmacology 0.63 0.15 0.39 0.22 0.35

Physiology 0.67 0.32 0.47 0.38 0.46

*values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measures of

Quality – AAMC

GQ

Year 1 courses Overall Satisfaction AY 2013-2014

Human Anatomy and Embryology II 4.57 Human Anatomy and Embryology I 4.33 Physiology-Renal 4.32 Biostatistics and Epidemiology 4.28 Metabolic Basis of Disease 4.23 Biochemical and Genetic Basis of Medicine 3.97 Neuroscience 3.90 Physiology-Endocrine 3.81 CTO 3.79 Basic Science of Microbial Disease 3.78 Virology 3.71 General Pathology 3.46 Physiology-Cardiovascular 3.45 Physiology-Respiration 3.38 Immunology 2.35

Measures of Quality – Course Reviews

scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality – Course Reviews

Biostats2010

(54%)*

Biostats 2011

(23%)*

Biostats 2012

(96%)*

Biostats 2013

(71%)*

Overall satisfaction of course 4.35 4.40 4.12 4.28

Overall usefulness of lectures 2.83 3.17 2.89 N/A

Overall usefulness of small groups 3.85 4.11 4.34 N/A

Overall usefulness of course materials 4.65 4.67 4.50 N/A

Congruence of assessment questions to material emphasized in course 4.48 4.61 4.48 4.69

*student participation rate on course evaluation

Measures of Quality – Student Comments

Summary of Student CommentsParticularly Well Done

• Overall organization, relevance, and clarity of course• Small groups, including ability to choose between

preparing ahead and working in class• Course notes, option to receive a binder

Could Have Been Better• Small groups could be more efficient• Notes could correlate better with lectures

Measures of Quality – Student Comments

Suggestions• Final lecture (screening) should not be required • Make lecture slides available before class• Utilize iPads to write on and project rather than

chalkboard• Visibility on Echo360• Utilize more screencasts• Provide only one facilitator per group• Train facilitators more

Measures of Quality – Student Comments

Discrepancy between those with a statistics background and those without“I have never taken a stats class before and I found that the notebook text was written for someone with a little bit more understanding than a beginner. There were a few concepts that were not fully explained (power, confidence intervals) that I figured out with the help of my small group professor and Khan videos. I think that this course should be for people who haven't taken stats, and it should be taught at that level. The class wasn't particularly difficult, it was just hard to be with a group of peers who have an extensive knowledge of the topic and to try to learn at a speed that worked for both of us.”

“I did not learn very much from this course as I've already taken stats and healthcare research design. I know multiple students have similar or more background and it might be worth restructuring sections or allowing students to test out, rather than having us complete what feels like "busy work" for a full term.”

Measures of Quality – Student Comments

Discrepancy between those with a statistics background and those without

Heavy background● Too slow, especially at beginning● Too much review● Not difficult enough● Small groups helpful, but should be

more efficient, shorter, or optional

No background● Appreciated slow pace● Liked beginning with the basics● Felt some things needed more

explanation● Small groups were extremely helpful

Suggestions● Sort groups based on previous stats experience (2)● Make small groups optional, especially for students with stats background (4)● Allow students to test out of course (4)

Summary regarding Measures of Quality• This course is consistently well-received by students

and rated in the “very good” to “excellent” range – the course director and her faculty are to be commended for their efforts

• Students do very well on Step I in this discipline• Students suggest minor adjustments to make the

course even better• The major issue that the course director should

address is how to meet the needs of two groups of students – those with a no prior knowledge of the material, and those with significant knowledge

Recommendations• The session objectives in Ilios and those distributed to the

students need to be consistent – the “old” set in the course materials needs to be replaced with the correct version that is in Ilios (it would also be a good idea for the course director to take this opportunity to review these objectives to make sure they still reflect the content that is covered in the course)

• The course director should consider the student suggestions and make changes where appropriate; one suggestion in particular (providing materials before class) should be addressed to comply with Y1 policy (i.e. class materials should be posted at least 24 hours before the scheduled class session)

Recommendations• The course director should consider how to meet the

needs of two different populations of students that take the course. Multiple ideas were discussed during the subcommittee meeting and we are confident there is a viable solution.

Action Plan (1)

1. Update objectives in the course materials that are not matched to those in Ilios

2. Upgrade lecture/notes in Lecture on Public Health Emergencies & Population Epidemiology

3. Request lecturers provide slides >=24 hours before4. Ask lecturers not to use chalk board but learn how

to “write” on slides at podium5. Revise power/sample size notes for clarity

Action Plan (2)Consider / discuss:

a) Adding a self-selected option for small group for students with background in the subject – journal club, discussion of a paper and its methods

b) Optional v mandatory small groupsc) Format & attendance at screening class which was

experimental format last year