Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High...
Transcript of Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High...
![Page 1: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/1.jpg)
ModuleECUE«AppliedAI»
AI&Biomedical:Bigdatainbioimagery
XavierDescombesMorphemeteamINRIA/I3S/iBV
![Page 2: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/2.jpg)
HighSpatialResolutionMultiscale
• Microscopyimages:– Spatialresolutioninx/y:lowerthan1µm– 2Dor3Ddatasets:uptoseveralhundredsofslices
Example:micebrainimageonlight-sheetmicroscopyX=0.75µm,Y=0.75µm,Z=1.99µm
6000x6000x1000voxelsVoxelscodedon16bitStudyon2channels+timecourse20-300Gbabout40brains
![Page 3: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/3.jpg)
HistopathologydataPyramidalstructureofdata(Whole_SlideImagingformat)
![Page 4: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/4.jpg)
Histopathologicaldata
(822x1.365) (13.152x21.840) (52.608x87.360)
Firstgoal:focusonsuspiciousareasMultiscaleapproach:
Firstanalysethelowresolutionimagetoconsideronlysmallareasonthehighresolutionimage(toreduceforexamplethesizeofaCNNfirstlayer)
![Page 5: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/5.jpg)
Histopathologydataanalysis
• Goal:classifyandgradethecancer• FocussonROI(tumorzones)• Medicaldecisionbasedonlocalpatterns• Histopathologistanalysis:
– ScreenthelowresolutionimagetodetectROIs– ZoomontheseROIs(andcomeback)->multiscaleanalysis->qualitativeanalysis
![Page 6: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/6.jpg)
Afirstmachinelearningtask
• ClassifyROI(tumorareas)
Tumorareas(ROI)InitialImage
![Page 7: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/7.jpg)
Challenge
§ Variabilitybetweenandwithinimages§ Noninformativeareas(fat,blood…)§ Hugedatasets(12Go≅100000pixelsperaxis)
![Page 8: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/8.jpg)
Patchesclassification
Tumor
![Page 9: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/9.jpg)
Pre-Processing:colordeconvolution
RGBimage Hchannel Echannel
![Page 10: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/10.jpg)
Reducedataset• Removepatcheswithfewcells
ImageRGB Hchannel Detectednuclie
32nuclei
120nuclei
![Page 11: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/11.jpg)
Reducedataset
![Page 12: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/12.jpg)
Featuresextraction:localbinarypatterns
Imagegrayvalues(P=8) RLBP
Rotationinvariance
![Page 13: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/13.jpg)
Classification:k-means
Hchannel EchannelRGBimage
Resultafternucleidetection
14 440 8700 … … … … … … 745RLBP-H
945 560 163 … … … … … … 12RLBP-EP=16,R=3(65.536x2patterns)
FindtheMostFrequentpatternsfromthetrainingimages
ClassificationK-NN
• Classesnumber:02(Tumor/Not-tumor)• Learning:10slidesè850patches• Test:05slides
![Page 14: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/14.jpg)
Resultonlearningset
Inputimage GroundTruth Result
![Page 15: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/15.jpg)
Resultontestset
Inputimage GroundTruth Result
![Page 16: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/16.jpg)
HighThroughputdata
PILOTSCREEN
Kinases and phosphatases (563)
RNA binding proteins (406) ~1000 out of 13000 genes
GFP-Imp+cells
highthroughputconfocalmicroscope
RNAi
GFP-Imp/DAPI
3millionsofimages!!!
![Page 17: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/17.jpg)
HighThroughputdataFirstconsideraPILOTscreen(subsampleofwellchosengenes):
Kinasesandphosphatases(563)
RNAbindingproteins(406)~1000outof13000genes
![Page 18: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/18.jpg)
ApipelineforanalysingthescreenSelection of healthy cells
Extraction of cell cytoplasm
Detection of Imp granules
Hit identification
segmentation machine learning
classification
healthy cells
RAW
IMA
GE
DAT
A B
ASE
segmentation
DAPI GFP-Imp
SPADE algorithm
GFP-Imp+ cells GFP-Imp particles
statistics clustering
Altered granule size, number, distribution ?
GFP-Imp
Disciplinesinvolved:biology,machinelearning,imageprocessing,database
![Page 19: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/19.jpg)
Machinelearningtask
• Classifythecellsw.r.t.thegranulepopulation• Features:number,size,spatialrepartition• Challenge:unknownnumberofclasses• Rejectionclass• Unbalancedclasses• Hugenumberofsamples:checkingdifficult
![Page 20: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/20.jpg)
Timesequences
![Page 21: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/21.jpg)
Timesequences
![Page 22: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/22.jpg)
Multipleparticlestracking
• Mainapproachesintwosteps:1) Objects(particles)detection2) Objects(particles)linking
• Particularcase:(forlowspeed)» Trajectoriesdetectionin(Space+Time)domain
![Page 23: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/23.jpg)
Thechallenges• Detection(seecorrespondingcourse)
• Appearance/Disappearanceofparticles
• Crossing• Occlusion
• Noise
• LocationVSshapedescriptor
![Page 24: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/24.jpg)
DetectandMatcht t+1
![Page 25: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/25.jpg)
Match:nearestneighbor
xi
yj
M(xi)=argmind(xi,yj)yj
![Page 26: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/26.jpg)
Unicityconstraintversus
Matchingmatrix: M(i, j) = 0 or 1X
i
M(i, j) = 0 or 1
Maximumvelocity
versus
d(xi, yj) > Vmax =) M(i, j) = 0
![Page 27: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/27.jpg)
Globaloptimization
argminM
X
i2I,j2J
d(xi, yj)M(i, j)
M(i, j) = 0 or 1
8iX
j
M(i, j) = 0 or 1
d(xi, yj) = f(||yj � xi||)Vmax V
f(V )
![Page 28: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/28.jpg)
Movementmodeling
• Brownianmotion:randommovement(bigparticleinafluid)
https://fr.wikipedia.org/wiki/Mouvement_brownien#Processus_d%E2%80%99Ornstein-Uhlenbeck
P (xt+1|xt) =1
2⇡p�2
exp
� (xt+1 � xt)2
2�2
�
![Page 29: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/29.jpg)
Deterministicspeedmodelxt+1 = xt + Vt(xt)dt+ d⌘t+1
Vt(xt)
⌘t+1
Deterministicmodel
Fluctuation
![Page 30: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/30.jpg)
Advantageofamodel
versus
Speed:learntfromamodelorestimatedfrompaststeps
![Page 31: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/31.jpg)
Gapfilling
t t+1 T+2
Graphmodel:MinimalPath
DA
LinkingCost
![Page 32: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/32.jpg)
Tracklets
v
Twosteps:Local(trackletsdetection),trackletsmergingPros:considertrajectoryand/orspeedmodels
![Page 33: Module ECUE « Applied AI » AI & Biomedical : Big data in bio imagery · 2019. 10. 7. · High Spatial Resolution Multiscale • Microscopy images : – Spatial resolution in x/y](https://reader033.fdocuments.us/reader033/viewer/2022051904/5ff65ca388f7c04bc107f732/html5/thumbnails/33.jpg)
Takehomemessage
• Donotconsiderhigherresolutionthanneeded(forspace,timeandintensity)
• Consideramultiscaleapproach• Adapttheprocessingtothesizeofdata(compromisebetweenaccuracyandcomputaitontime)
• Considerparrallelprograming