Convolutional Neural Networks For Modeling Temporal...

Convolutional Neural Networks For Modeling Temporal Biomarkers And

Disease Predictions

Narges Razavian New York University Langone Medical Center

GTC 2017

In collaboration with: David SontagPhD, Saul BleckerMD, Ann-Marie SchmidtMD, Enrico BertiniPhD, Rahul Krishnan, YD Choi, Josua Krause, Somesh Nigam, Aaron Smith-McLallen, Ravi Chawla

Deep learning progress Healthcare world getting digital

Parallel Developments

EHR adoption by healthcare centers in the US

Error rate on Image-Net object recognition challenge

What is captured in the EHR?

Source: healthcare.gov

Healthcare has joined the data-rich world

Moving from Treatment to Prevention

Challenges: Each Individual has a different ‘healthy’ baseline.

- Temporal Patterns/Trends are predictive Each biomarker varies at a different speed in our bodies Measurements are sparse, asynchronous and correlated Many correlated outcomes are observed per patient

- Can we leverage this correlation?

Biomarkers and Outcomes

Biomarkers measurements

over time

Phenotype (diseases) over time

over time

Step 1 Learn each biomarker from other biomarkers time-series

Kernel Regression

Observations

Time Not Observed Want to estimate

Kernel Regression

Observations

E[X(v)]= xP(x |∫ t = v,Xtrain )dx

E[X | t = v,Xtrain ]= x∫ P(x, t = v | Xtrain)P(t = v | Xtrain)

E[X | t = v,Xtrain ]= x∫K(x − xi,v− ti )

xi ,ti

K(v− ti )ti

Kernel Regression

Observations

E[X(v)]= xP(x |∫ t = v,Xtrain )dx

E[X | t = v,Xtrain ]= x∫ P(x, t = v | Xtrain)P(t = v | Xtrain)

E[X | t = v,Xtrain ]= x∫K(x − xi,v− ti )

xi ,ti

K(v− ti )ti

E[X | t = v,Xtrain ]=(K ⊗ Xtrain )(v)

(K ⊗ I(Xtrain :Observed))(v)

Use convolution framework to LEARN those kernels

We can learn the kernel (No need for parametric forms and cross validations) Easily extendible to multivariate!

Unsupervised: All needed is (asynchronous) sequence of observations. Fast to train. Fast to apply.

Benefits

Data:30KIndividualsfromtheoriginaltrainingset.Datasetsplitequallybetweentrain,testandvalidateset.Loss:MSE.Trainandevaluateonlyon(lab,person)withmorethan1observaGon.

Mul$variateKernelslearnedforeachinputdimension(total18)

More details in our ICLR paper

Narges Razavian, David Sontag Temporal Convolutional Neural Networks for Diagnosis from Lab Tests http://arxiv.org/abs/1511.07938 Open Source code available (torch/lua implementation): https://github.com/clinicalml/deepDiagnosis

Step 2 Predict 200+ correlated outcomes using multi-resolution convolutional neural networks and multi-task learning

Multi-Resolution Convolution Networks The Architecture - model (1)

Multi-Resolution Convolution Networks The Architecture - model (2)

Prediction AUCs on the held-out test set

More details in our JMLR paper

Narges Razavian, Jake Marcus, David Sontag, Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests JMLR, 2016 http://arxiv.org/abs/1608.00647 Open Source code available (torch/lua implementation): https://github.com/clinicalml/deepDiagnosis

Following up in clinical world •  Prediction models built and deployed for

–  Nurse calls and home visits for 250,000+ NYUMC patients at high risk for a number of these outcomes

–  Improved documentation in EHR •  Automation of mandatory visits/screening/follow-ups •  Best practice alerts •  Reimbursement for intense lifestyle management programs

•  Extending to broader outcomes and domains

New York University (i2b2) Database

New York University (i2b2) Database Nuclear Medicine Procedures

Magnetic Resonance Imaging

Conclusions •  Applications of deep learning in healthcare are unlimited

•  Unsupervised learning + back-propagation + deep learning can recover biomarker models from asynchronous high-dimensional time-series data

•  Multi-task learning benefits prediction tasks with smaller datasets.

Thanks!

Questions/comments: Narges.Razavian@nyumc.org

Open Source Package: https://github.com/clinicalml/deepDiagnosis

Convolutional Neural Networks For Modeling Temporal...

Documents

Transcript of Convolutional Neural Networks For Modeling Temporal...

Temporal Convolutional Neural Networks for Radar Micro ...

Convolutional Neural Networks

Convolutional Neural Network

Convolutional Neural Fabrics - arXiv

Convolutional Neural Networks - Virginia Techjbhuang/teaching/ece5554-4554/fa17/... · Convolutional Neural Networks Computer Vision ... ImageNet Classification with Deep Convolutional

Constrained Convolutional Neural Networks for …vgg/rg/slides/ccnn1.pdf · Constrained Convolutional Neural Networks for Weakly Supervised Segmentation ... [CCNN] Convolutional Neural

Shepard Convolutional Neural Networks - Neural Information …papers.nips.cc/paper/5774-shepard-convolutional-neural... · 2016-02-16 · Shepard Convolutional Neural Networks (ShCNN)

Convolutional Neural Networks Analyzed via Convolutional ... · PDF fileConvolutional Neural Networks Analyzed via Convolutional Sparse Coding ... Deep Learning, Convolutional Neural

Convolutional Neural Network · Convolutional Neural Networks (CNN) •A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital

Temporal Convolutional Neural Network for the Classification ...Article Temporal Convolutional Neural Network for the Classiﬁcation of Satellite Image Time Series Charlotte Pelletier

Using Support Vector Machines, Convolutional … Support Vector Machines, Convolutional Neural Networks and ... Using Support Vector Machines, Convolutional Neural ... orcircuit ,andmuchANNresearch

Probabilistic Forecasting with Temporal Convolutional ... · Probabilistic Forecasting with Temporal Convolutional Neural Network Yitian Chena, Yanfei Kangb,, Yixiong Chenc, Zizhuo

Deep Convolutional Neural Network for Image …papers.nips.cc/paper/5485-deep-convolutional-neural...Deep Convolutional Neural Network for Image Deconvolution Li Xu ∗ LenovoResearch

CONVOLUTIONAL NEURAL NETWORKS: MOTIVATION, …

TinyRadarNN: Combining Spatial and Temporal Convolutional ...TinyRadarNN: Combining Spatial and Temporal Convolutional Neural Networks for Embedded Gesture Recognition with Short Range

Interpretable Convolutional Neural Networksopenaccess.thecvf.com/content...Convolutional_Neural_CVPR_2018_… · Interpretable Convolutional Neural Networks Quanshi Zhang, Ying Nian

Hyperbolic Graph Convolutional Neural Networkspapers.nips.cc › paper › 8733-hyperbolic-graph-convolutional-neural-networks.pdfGraph Convolutional Neural Networks (GCNs) are state-of-the-art

Visualizing and Understanding Convolutional Neural ... - arXiv and Understanding Convolutional Neural ... - arXiv

A Convolutional Neural Network Cascade for Face … · A Convolutional Neural Network Cascade for Face Detection ... using a GPU, and achieves state ... Convolutional Neural Network

Convolutional Neural Networks - arXiv