SYMPOSIUM ON MEDICAL CURRICULUM HARMONIZATION THE OPTIMED PROJECT.

SYMPOSIUM ON MEDICAL CURRICULUM

HARMONIZATION

THE OPTIMED PROJECT

SYMPOSIUM ON MEDICAL CURRICULUM HARMONIZATION

INVITED SPEAKERSJulie Bienertová Vašků (FMed MU)

Comprehensive innovation of medical education

Martin Komenda (IBA MU)

OPTIMED portal platform in practical use

Jan Švancara (IBA MU)

Advanced data-analytical reports mined from medical curriculum mapping tools


Comprehensive Innovation of Medical EducationJ. Bienertová Vašků, M. Komenda, J. Štěrba,

J. Mayer, L. Dušek


Motivation

• The need for a well-balanced medical curriculum

• Re-engineering education-associated activities• Providing a clear overview of curriculum structure• Simple in-depth curriculum management

• Proven methodological background

• Outcome-based paradigm

Curriculum – preparation for practice

Students

Content – what we teach

Strategies – how we teach

Evaluation – do we teach well?

Educational environment – where we teach


Educational process objectives

1

2

3

4

5

The graduate will be able to…

Practitioner, scientist….

Ability to interpret X-ray image

Student capable of interpreting x-ray image of the knee

Student capable of describing typical signs of arthrosis on knee X-ray

General educational target

Target role

Curriculum outcomes

Detailed curriculum outcomes

Study outcomes

Source: Scottish Doctor


Primary objectives

• Comprehensive innovation of the General Medicine field of study

• Mapping the existing state at the Faculty of Medicine

• Improving orientation and transparency across learning with the use of suitable ICT technologies

• Providing better links secondary education outcomes


Target group

• Faculty of Medicine academic employees

• Teachers• Curriculum designers• Guarantors• Supervisors• Faculty management

• Students

Content harmonizationof the medical curriculum

?

The OPTIMED project


Medical section overview

Internal medicine Surgical sciences Theoretic sciences DIA a neurosciences

Formal structure of the medical curriculum

• Surgical sciences

Section

• VLCH0731c - Surgery I - practice

Course

• Pediatric Urology

Learning unit

• Student describes most common manifestations, diagnosis, surgery and prognosis of Wilms tumour

Learning outcome

OPTIMED in numbers

Total number of General Medicine

Modules 4Courses 144Learning units 1,347Learning outcomes 8,063Teachers and guarantors 385Students over 2,000

Courses according to sections


Curriculum harmonization planning model

1. Setting up the composition of curriculum in accordance with formal structure

2. Definition of the medical curriculum (learning units, learning outcomes and all metadata)

3. Vertical harmonisation covering verification and further discussion within the individual module


Curriculum harmonization planning model

4. Horizontal harmonisation based on follow-up discussions across all modules

5. Evaluation of all available content with the use of multidimensional standardised assessment forms

6. Creation of educational content according to the defined learning outcomes

Multidimensional curriculum evaluation and validation

Student opinion

Teachers’ opinions

Methodologists’ opinions

Opinion of thesenior guarantor

Analysts’ opinions

Student evaluation

Teacher evaluation

Methodological evaluation


Curriculum designer evaluation

• Set of modeling terms and learning outcomes

• Inspiration by proven standards

• …

Analytical overview

Google analytics reportApril 1, 2014 to date

Possible uses of OPTIMED

OPTIMED - optimalizovaná výuka všeobecného lékařství: http://med.muni.cz/optimed

OPTIMED

Definition of requirements

for study

Educational resources

Evaluation of graduates

Selection of candidates for study

Curriculum definition

Evaluation and changes to curriculum

Educational management

and QA


Conclusion

1. Providing a unique description of the study content in the field of medical education in the Czech Republic

2. Defining the medical curriculum (learning units, learning outcomes and all metadata) for the General Medicine field of study

3. Definition continuity of information throughout the study

4. Identifying inconsistencies in studies, providing tools for correcting identified deficiencies

5. Defining connections between the General Medicine field of study curriculum and secondary education in the Czech Republic

6. Providing a methodological platform for study description


Conclusions

1. Clarifying connections between prerequisite fields of study

2. Monitoring the sequence of specific issues throughout the entire duration of studies (Identifying missing and duplicit information)

3. Providing a link to the MU Information System by interconnecting curricula records and study materials


Thank you for you attention


OPTIMED portal platform in practical

useM. Komenda, D. Schwarz, J. Bienertová Vašků,

J. Štěrba, J. Mayer, L. Dušek


Outline

• Engineering background

• OPTIMED portal platform – modular architecture

• Use case: How to work with OPTIMED


Motivation

• To propose a curriculum planning model

• To support an outcome-based approach

• To develop a robust web-oriented platform


Formal description of medical curriculum • The base of OPTIMED platform

• No other formal specification exists

• Independent on technologies

Formal description of medical curriculum

Engineering background

Modular architecture


Curriculum browser

Students• Discover what knowledge shall be obtained during studies• Identify which topics will be covered repeatedly• Find clear information about overlaps (desirable or not)

Lecturers• Easy way of

• a describing their lessons clearly• a browsing curriculum data of all available courses

Management• On-line transparent overview of curriculum• Decision support of curriculum building and innovation

Reporting tools


OPTIMED in practical use

Coloured pictographs make the first look more transparent.

Internal medicine

Theoretic sciences

Surgical sciences

DIA and neurosciences

Which topics are covered in the course of Intensive care medicine?

FILTER

I found- who are teachers and guarantors,- how long the unit takes,- why is the unit taught,- what are the most important terms,- the list of learning outcomes,- recommended learning materials.

But still, something is missing. Let’s provide a feedback.

FILTER

Where is blood pressure taught?

FILTER


Future visions

• Sharing of our know-how• Curriculum planning model, database structure,

organisation of educational metadata, OPTIMED portal platform

• Curricula comparison• NLP, machine learning, and data mining techniques

Special thanks to…


Thank you for your [email protected]


Advanced data-analytical reports mined from medical

curriculum mapping tools

J. Švancara, M. Komenda, J. Jarkovský, L. Dušek


Motivation behind

• Descriptive overview of medical curriculum

• Supporting materials for the evaluation and assessment for the global in-depth inspection

• Finding the potentially problematic areas (overlaps and imperfections)

• Construct comprehensive reports


Data analysis – theoretical background• Data analysis is process of getting information from data

• Data analysis helps us to understand complex problems and provide support for decision making.

• IBA MU is oriented on application of modern information technologies in analysis of extensive and complex data sets from:• clinical research (e.g. clinical trials)• environmental sciences (e.g. chemical monitoring) • and also educational research (e.g. analysis of the admission

procedure)

Multidimensional curriculum evaluation and validation

Student opinion

Teachers’ opinions

Methodologists’ opinions

Opinion of thesenior guarantor

Analysts’ opinions


Expert evaluation

• Target group: senior curriculum designers

• Aims• Critical assessment of modeling terms and outcomes• Professional feedback towards terms occurence in

medical curriculum• MeSH standardization• Transparent overview


Learning unit

Section SemesterMeSH key

wordsModeling

terms

Modeling term

Expert evaluation 13 criteria

MeSH key words

Class CategoryTree

Structure

Learning unit

MeSH key

words

Modeling terms

Section Semester Class CategoryTree

StructureExpert

evaluation

Input data[N = 1346 learning units]

[161 modelling terms evaluated by 13 experts]

[Complete MeSH key words dictionary]

Final dataset


Medical Subject Headings

• Standardized biomedical dictionary

• Contains 26 142 entries with over 54 000 links

• Regular updates of the Czech mutation

• Straight links between MeSH key words and learning units

Sample of data


Primary goals

• Data processing and analysis

• Mining novel and potentially useful patterns

• Reduction of the time spent on global curriculum overview and systematic evaluation

• Making outputs more transparent and readable


Data-analytical reports mined from medical curriculum data

Relation between learning units and MeSH key words

1 2 3 4 50%

10%

20%

30%

40%

50%

60%

11.9

% 17.5

%

17.5

%

15.2

%

37.7

%

8.6%

14.2

% 20.5

% 26.0

% 30.7

%

5.4%

15.5

%

22.6

%

23.4

%

33.1

%

3.2%

10.8

% 15.7

% 19.8

%

50.5

%

Surgical sciences

Internal medicine

DIA and neurosciences

Theoretic sciences

1346 learning units were described by:• 3224 unique MeSH key words• 5090 total number of MeSH key words

Perc

ent o

f lea

rnin

g un

its

Number of MeSH key words in learning unit description

MeSH key words frequency

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 310

500

1,000

1,500

2,000

2,500

2334

528

182

81 37 18 11 10 6 3 5 1 1 1 1 1 1 2 1

2334 MeSH key words was used for just a one single learning unit

Num

ber o

f MeS

H k

ey w

ords

Number of learning units decribed by given MeSH key word

MeSH key word tree structureMeSH key words used in describtion of learning units are mostly for 3-5 level of MeSH Tree Structure.

Perc

ent o

f lea

rnin

g un

its

Tree level

1 2 3 4 5 6 7 8 9 100%

5%

10%

15%

20%

25%

30%

35%

40%

45%

4.5%

21.2

%

38.4

%

23.9

%

8.6%

2.8%

0.3%

0.3%

0.0%

0.0%0.8%

4.2%

16.7

%

27.1

%

25.6

%

16.1

%

6.3%

2.5%

0.6%

0.1%

MinimalMaximal

Average tree level Minimum Maximum

Surgical sciences 3.27 4.30

Internal medicine 3.26 4.62

DIA and neurosciences 3.20 4.55

Theoretic sciences 3.09 4.74


Modeling terms – criteria

• Explanation of a term in the contents of studies • Adequate classification of a term during studies • Evaluation of continuity • Prerequisites • Overall proportionality • Relation to clinical practice • Consistency of representation in studies • Nomenclature • Time burden • Continuity in study materials and tools • Continuity in learning outcomes • Continuity in learning units • Explanation of theoretical background


Modeling outcomes – criteria

• Interconnection with learning units

• Completeness of learning units

• Theoretical background for learning outcomes

• Relation to clinical practice

• Interconnection between theory and clinical practice

• Saturation with important terms

• Time allocation

Modeling terms – expert evaluation results

Explan

ation of a

term

in th

e contents

of stu

dies

Adequate

classi

fication of a

term

during s

tudies

Evaluati

on of continuity

Prerequisit

es

Overall p

roporti

onality

Relation to

clinica

l practi

ce

Consisten

cy of r

epres

entation in

studies

Nomencla

ture

Time b

urden

Continuity in

study m

ateria

ls and to

ols

Continuity in

learn

ing outcomes

Continuity in

learn

ing units

Explan

ation of t

heoreti

cal b

ackgro

und75

80

85

90

Expe

rt e

valu

ation

(sca

le 0

-100

)

Modeling outcomes – expert evaluation results

Interconnection with

learning units

Completeness of le

arning units

Theoretical background for le

arning outcomes

Relation to cli

nical practice

Interconnection betw

een theory and cli

nical practice

Saturation with im

portant te

rms

Time allocation

60

65

70

75

80

85

90

95

100

Relation to practice is weak point of modeling outcomes

Expe

rt e

valu

ation

(sca

le 0

-100

)

Modeling terms - typology

Variability – Std. Deviation of expert evaluation

Qua

lity-

mea

n of

exp

ert e

valu

atin

Preclinical studiesClinical studies

Good rating

Ambivalent

Poor rating

Modeling outcomes - typology

Good rating

Ambivalent

Poor rating

Variability – Std. Deviation of expert evaluation

Qua

lity-

mea

n of

exp

ert e

valu

atin


Modeling terms – the best 10

• Blood• ECG• Delivery• Reflex• Obesity• Inflammation• MRI• Vitamins• Health• Skin


Modeling outcomes– the worst 10

• Operability• Regeneration of nervous system• Tissue antigens• Respiratory failure• Transtentorial herniation• Brain centres• Brain perfusion• Treatment complications• Autoinflammatory diseases• Penumbra


Association mapsDIA and neurosciences

179 learning units are described by single and uniqe MeSH key word

Good rating

Ambivalent

Poor rating

• Association is reciprocal

• Larger circle represents higher sum of association coefficients

DIA and neurosciences: inflammation

Surgical sciences: inflammation

Internal medicine: inflammation

Theoretic sciences: inflammation


Conclusions

• Evaluation of modeling terms• Evaluation of all 13 areas highly correlate each other• Typology of modeling terms is based on average score

and variability among evaluators (in Good rating category are terms with high score and low variability).

• There is just few terms with poor rating. These terms have low frequency in learning units description.

• Evaluation of modeling outcomes• Poor ration of modeling outcomes is very rare.• Relation to practice is weak point of modeling outcomes.


Conclusions

• Learning units are described by 5090 MeSH key words.

• 179 learning units are described by single and uniqe MeSH key word.

• 3224 MeSH key words were used just once.

• More than 60% learning units is described by 4 or 5 MeSH key words.


Discussion

SYMPOSIUM ON MEDICAL CURRICULUM HARMONIZATION THE OPTIMED PROJECT.

Documents

Transcript of SYMPOSIUM ON MEDICAL CURRICULUM HARMONIZATION THE OPTIMED PROJECT.