Validation of a Software Based Clinical Decision Support ... · Access to expert, evidence based...

1
Access to expert, evidence based clinical decision making is crucial in maximizing the outcome of women with breast cancer, but it is a scarce resource, especially in developing countries. The Navya Expert System is a patented, software based clinical decision support system that exhaustively searches and assimilates relevant medical literature and guidelines to make specific therapeutic recommendations for individual patients based on their clinical data. This study is a retrospective validation of Navya Expert Systems output against tumor board decisions of a multidisciplinary group of expert breast cancer clinicians working in a tertiary care oncology center in India With structured clinical data as the input, the patented Navya Expert System queries its structured databases of evidence and guidelines indexed per Navyas ontology of decision criteria, and returns as output treatment decisions most applicable to a given unique patient. The objective of the study was to test the results of the automated system against tumor board decisions made by expert treating oncologists. The automated system is inherently scalable and aims to eventually enable patients and physicians worldwide to increase their access to evidence based and expert treatment decisions. Background P4-16-01 A total of 76 patients involving 224 major and 224 minor therapeutic decisions were included in the study. Navya Expert System's output was concordant with the tumor board or expert review 224/224 major decisions (100%, 95% CI 99.6%-100%) 221/224 minor decisions (98.6%, 95% CI 97.1%-100%) Navya Expert System's output was concordant with the tumor board alone 210/224 major decisions (93.75%, 95% CI 90.6%-96.9%) 160/224 minor decisions (71.4%, 95% CI 65.5%-77.3%) Most common reasons for discordance were Non-prescription of HER2 targeted therapy by the tumor board due to financial constraints Non-use of adjuvant radiotherapy for 1-3 node positive patients.(as tumour board decisions were based on guidelines from 2012) Of the 64/224 Navya Expert System decisions discordant with the tumor board, only three Navya decisions were finally deemed discordant with expert clinical practice Future Trials MAJOR DECISIONS MINOR DECISIONS NACT or Upfront surgery BCT or MRM Adjuvant chemotherapy or not Anthracyclines or Taxanes or combination Adjuvant Radiation therapy or not Nodal RT/IMRT or no radiation therapy Access to Evidence Based Expert Decisions = Maximize Outcomes Objective Methodology Results Navya decisions were compared to tumor board decisions and adjudicated by experts . A retrospective trial was conducted to compare the Navya Expert System decisions to tumor board decisions made at a tertiary care cancer centre (Tata Memorial Centre, Mumbai) in India. Cases of women with non-metastatic breast cancer treated at Tata Memorial Center, for whom treatment decisions involved therapeutic complexities such as two or more treatment options , were included in the trial. Clinical and pathological data were abstracted from the case files as per Navya’s decision criteria, eg: Age, ER/PR /Her2neu status, Tumor Size, Number of positive lymph nodes, Co-Morbidity, etc Patient preferences, and financial and logistic constraints were not factored into the decision criteria. Data was processed through Navya’s automated, patented Evidence Engine and Guidelines Engine. Navya treatment decisions were classified as follows: © Tata Memorial Hospital, Mumbai, India & Navya Network , Cambridge, MA This presentation is the intellectual property of the author/presenter. Contact at [email protected], [email protected] for permission to reprint and/or distribute Results Conclusion A conjoint analysis based decision aid to assess risk/benefit tradeoffs for patients will be prospectively validated to include as a resource in decision making. This trial will test the hypothesis that factoring in patients preferences in evidence based expert decisions will result in further improved outcomes. Commonly available clinical data + Navya Expert System = Expert Treatment Decisions Navya decisions are concordant with expert clinical practice i.e. concordant with the tumor board and experts at a tertiary care cancer center Navya Expert System can increase global access to evidence based expert treatment decisions San Antonio Breast Cancer Symposium - December 9-13, 2014 Validation of a Software Based Clinical Decision Support System for Breast Cancer Treatment in a Tertiary Care Cancer Center in India NS Nair 1 , S Gupta 1 , N Ramarajan 2 , G Srivastava 2 , V Parmar 1 , A Munshi 1 , S Vanmali 1 , V Vanmali 1, R Hawaldar 1 , RA Badwe 1 1. Tata Memorial Centre, Mumbai, India, 2. Navya Network, Cambridge, MA DECISION (N) TUMOR BOARD + EXPERT ADJUDICATOR Major Conc.(CI) TUMOR BOARD + EXPERT ADJUDICATOR Minor Conc.(CI) TUMOR BOARD Major Conc. (CI) TUMOR BOARD Minor Conc. (CI) Total 224 100% (0.41) 98.66% (1.5) 93.75% (3.16) 71.42% (5.91) NACT/ Sx (108) 100% (0.59) 99.07% (1.80) 94.44% (4.32) 79.62% (7.59) NACT Her2neu -ve (14) 100% (1.65) 92.8% (13.49) 100% (1.65) 64.28% (25.09) NACT Her2neu+ (12) 100% (1.78) 100% (1.78) 100% (1.78) 8.33% (15.63) Surgery (82) 100% (0.68) 100% (2.52) 92.68% (0.68) 92.68% (0.68) Adjuvant chemotherapy (38) 100% (1) 94.73% (7.09) 100% (0.31) 31.57% (14.77) Adjuvant chemotherapy Her2neu –ve (18) 100% (1.4) 100% (1.43) 100% (1.46) 61.11% (22.52) Adjuvant chemotherapy Her2neu + (20) 100% (1.38) 90% (13.14) 100% (1.38) 5% (9.55) Adjuvant radiotherapy (66) 100% (0.76) 100% (0.76) 87.87% (7.87) 77.27% (10.11) RT N0 (3) 100% (3.57) 100% (3.57) 100% 93.57) 100% (3.57) RT 1-3 LN + (41) 100% (0.96) 100% (0.96) 80.48% (12.13) 65.85% (14.51) RT 4+ LN (22) 100% (1.32) 100% (1.32) 100% (1.32) 95.45% (8.70) Adjuvant hormone therapy (12) 100% (1.78) 100% (1.78) 100 % (1.78) 91.66% (15.63)

Transcript of Validation of a Software Based Clinical Decision Support ... · Access to expert, evidence based...

Page 1: Validation of a Software Based Clinical Decision Support ... · Access to expert, evidence based clinical decision making is crucial in maximizing the outcome of women with breast

Access to expert, evidence based clinical decision making is crucial in

maximizing the outcome of women with breast cancer, but it is a scarce

resource, especially in developing countries. The Navya Expert System is a

patented, software based clinical decision support system that exhaustively

searches and assimilates relevant medical literature and guidelines to make

specific therapeutic recommendations for individual patients based on their

clinical data. This study is a retrospective validation of Navya Expert System’s

output against tumor board decisions of a multidisciplinary group of expert

breast cancer clinicians working in a tertiary care oncology center in India

With structured clinical data as the input, the patented Navya Expert System

queries its structured databases of evidence and guidelines indexed per Navya’s

ontology of decision criteria, and returns as output treatment decisions most

applicable to a given unique patient. The objective of the study was to test the

results of the automated system against tumor board decisions made by expert

treating oncologists. The automated system is inherently scalable and aims to

eventually enable patients and physicians worldwide to increase their access to

evidence based and expert treatment decisions.

Background

P4-16-01

A total of 76 patients involving 224 major and 224 minor therapeutic decisions were

included in the study.

Navya Expert System's output was concordant with the tumor board or expert review

• 224/224 major decisions (100%, 95% CI 99.6%-100%)

• 221/224 minor decisions (98.6%, 95% CI 97.1%-100%)

Navya Expert System's output was concordant with the tumor board alone

• 210/224 major decisions (93.75%, 95% CI 90.6%-96.9%)

• 160/224 minor decisions (71.4%, 95% CI 65.5%-77.3%)

Most common reasons for discordance were

• Non-prescription of HER2 targeted therapy by the tumor board due to financial

constraints

• Non-use of adjuvant radiotherapy for 1-3 node positive patients.(as tumour board

decisions were based on guidelines from 2012)

Of the 64/224 Navya Expert System decisions discordant with the tumor board, only

three Navya decisions were finally deemed discordant with expert clinical

practice

Future Trials

MAJOR DECISIONS MINOR DECISIONS

NACT or Upfront surgery BCT or MRM

Adjuvant chemotherapy or not Anthracyclines or Taxanes or combination

Adjuvant Radiation therapy or not Nodal RT/IMRT or no radiation therapy

Access to Evidence Based Expert Decisions = Maximize Outcomes

Objective Methodology Results

• Navya decisions were compared to tumor board decisions and adjudicated by

experts .

A retrospective trial was conducted to compare the Navya Expert System decisions to

tumor board decisions made at a tertiary care cancer centre (Tata Memorial Centre,

Mumbai) in India. Cases of women with non-metastatic breast cancer treated at Tata

Memorial Center, for whom treatment decisions involved therapeutic complexities

such as two or more treatment options , were included in the trial.

Clinical and pathological data were abstracted from the case files as per Navya’s

decision criteria, eg: Age, ER/PR /Her2neu status, Tumor Size, Number of positive

lymph nodes, Co-Morbidity, etc

Patient preferences, and financial and logistic constraints were not factored into the

decision criteria.

Data was processed through Navya’s automated, patented Evidence Engine and

Guidelines Engine.

Navya treatment decisions were classified as follows:

© Tata Memorial Hospital, Mumbai, India & Navya Network , Cambridge, MA This presentation is the intellectual property of the author/presenter. Contact at [email protected], [email protected] for permission to reprint and/or distribute

Results

Conclusion

A conjoint analysis based decision aid to assess risk/benefit tradeoffs for patients

will be prospectively validated to include as a resource in decision making. This trial

will test the hypothesis that factoring in patient’s preferences in evidence based

expert decisions will result in further improved outcomes.

• Commonly available clinical data + Navya Expert System

= Expert Treatment Decisions

• Navya decisions are concordant with expert clinical practice i.e. concordant with

the tumor board and experts at a tertiary care cancer center

• Navya Expert System can increase global access to evidence based expert

treatment decisions

San Antonio Breast Cancer Symposium - December 9-13, 2014

Validation of a Software Based Clinical Decision Support System for Breast Cancer Treatment in a Tertiary Care Cancer Center in India

NS Nair1, S Gupta1, N Ramarajan2, G Srivastava2, V Parmar1, A Munshi1, S Vanmali1, V Vanmali1, R Hawaldar1, RA Badwe1

1. Tata Memorial Centre, Mumbai, India, 2. Navya Network, Cambridge, MA

DECISION (N)

TUMOR BOARD + EXPERT

ADJUDICATOR Major Conc.(CI)

TUMOR BOARD + EXPERT

ADJUDICATOR Minor Conc.(CI)

TUMOR BOARD Major Conc.

(CI)

TUMOR BOARD Minor Conc.

(CI)

Total 224 100% (0.41) 98.66% (1.5) 93.75% (3.16) 71.42% (5.91)

NACT/ Sx (108) 100% (0.59) 99.07% (1.80) 94.44% (4.32) 79.62% (7.59)

NACT Her2neu -ve (14)

100% (1.65) 92.8% (13.49) 100% (1.65) 64.28% (25.09)

NACT Her2neu+ (12)

100% (1.78) 100% (1.78) 100% (1.78) 8.33% (15.63)

Surgery (82) 100% (0.68) 100% (2.52) 92.68% (0.68) 92.68% (0.68)

Adjuvant chemotherapy (38)

100% (1) 94.73% (7.09) 100% (0.31) 31.57% (14.77)

Adjuvant chemotherapy Her2neu –ve (18)

100% (1.4) 100% (1.43) 100% (1.46) 61.11% (22.52)

Adjuvant chemotherapy Her2neu + (20)

100% (1.38) 90% (13.14) 100% (1.38) 5% (9.55)

Adjuvant radiotherapy (66)

100% (0.76) 100% (0.76) 87.87% (7.87) 77.27% (10.11)

RT N0 (3) 100% (3.57) 100% (3.57) 100% 93.57) 100% (3.57)

RT 1-3 LN + (41) 100% (0.96) 100% (0.96) 80.48% (12.13) 65.85% (14.51)

RT 4+ LN (22) 100% (1.32) 100% (1.32) 100% (1.32) 95.45% (8.70)

Adjuvant hormone therapy (12)

100% (1.78) 100% (1.78) 100 % (1.78) 91.66% (15.63)