Neural Network Segmentation and Validation
description
Transcript of Neural Network Segmentation and Validation
![Page 1: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/1.jpg)
Neural Network Segmentation and Validation
Nicole M. GroslandVincent A. Magnotta
![Page 2: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/2.jpg)
Objective
• To develop tools to automate bony structure mesh definitions suitable for patient-specific finite element contact analyses.
– Further, automate the identification of the structures of the upper extremity (including hand/fingers, wrist, elbow and shoulder) using a neural network.
![Page 3: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/3.jpg)
Specific Aims
• Aim 1: Integrate and enhance a set of novel and robust hexahedral mesh generation algorithms into the NA-MIC toolkit.
• Aim 2: Further automate these modeling capabilities by developing tools for automated image region identification via neural networks.
• Aim 3: Validate geometry of models using cadaveric specimens and three-dimensional surface scans
![Page 4: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/4.jpg)
Imaging Protocol• 15 cadaveric specimens were acquired and imaged
• CT images, Siemens Sensation 64 CT scanner (matrix = 512x512, KVP = 120).
– 0.34-mm in-plane resolution– 0.4 mm slice thickness
• MR images: Siemens 3T Trio scanner– PD weighted images – 2D FSE
• TE=12ms, TR=7060ms• Resolution=0.5x0.5mm• Slice Thickness = 1.0mm• Matrix=512x512
– T1 weighted images – 3D MP-RAGE• TE=3.35ms, TR=2530ms, TI=1100ms• Resolution=0.6x0.6x0.5mm• Matrix=384x384x96
• Post-processing via BRAINS2– Spatially normalized – Resampled to 0.2-mm3 voxels
![Page 5: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/5.jpg)
Manual Segmentation
• Two trained technicians (Tracer1 and Tracer2) manually traced twenty-one phalanx bones (index)
– the distal, middle, and proximal bones
• Relative overlap:
• Records maintained of tracing times
( 1 2)
( 1 2)
Volume Tracer TracerRelative Overlap
Volume Tracer Tracer
![Page 6: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/6.jpg)
Neural Network Data
Spherical Coordinates
Gradient ValuesArea Iris ValuesProbability Map Values
Input Vector: {PS1, PS2, Sα, Sβ, Sγ, G-4, … G4, A1, … A12}
Mask Values
Output Vector: {MS1, MS2}
![Page 7: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/7.jpg)
Neural Network Configuration
Output Layer test
Calculated Error
Backpropagation
Input Layer
Hidden Layer
![Page 8: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/8.jpg)
Neural Network Training
• 10 subjects used to train the neural network– Subjects all registered to atlas dataset– Manual segmentations used to define
probability information– 200,000 input vectors x 250 iterations
• 5 subjects used to evaluate validity and reliability of network
![Page 9: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/9.jpg)
3D Laser Scanner
• 3D Laser scanners have been used for rapid prototyping and to non- destructively image ancient artifacts
• Roland LPX-250 Scanner Obtained– Planar and rotary scanning modes– 0.008 inch resolution in planar mode– Objects up to 10 inches wide and 16 inches tall can
be scanned– Reverse modeling software tools
![Page 10: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/10.jpg)
LPX-250 Laser Scanner
![Page 11: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/11.jpg)
Finger Dissection
• Phalanx and metacarpal bones removed– Care taken to avoid tool marks on the bones
• De-fleshing process outlined by Donahue et al (2002) was utilized– Bones allowed to soak in a 5.25% sodium
hypochlorite (bleach) solution for 6 hours
• Degreased via a soapy water solution• Thin layer of white primer was used to coat the
bony surfaces
![Page 12: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/12.jpg)
![Page 13: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/13.jpg)
SC05030303RMD05042226LMD05010306RCA05042125L
![Page 14: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/14.jpg)
Registration of Surfaces
• Surface scans origin shifted to center of mass and reoriented to have the same orientation as the CT data
• Surfaces registered using a rigid iterative closest point algorithm
• Compute Euclidean distance between the surfaces
![Page 15: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/15.jpg)
a b c d
Specimen CA05042125L
Manual (red) and ANN (blue) ROI definitions
![Page 16: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/16.jpg)
Manual Segmentation
• Relative overlap (Tracer1 and Tracer2) – 0.89 for the three bones. – Individual bones
• Proximal – 0.91• Middle – 0.90• Distal bones – 0.87
• The average time required to manually segment the bones of the index finger was 50.9 minutes, ranging from 39 to 63 minutes.
![Page 17: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/17.jpg)
Subject ID Proximal Overlap
Middle Overlap
Distal Overlap
Index Finger
Overlap
CA05042124R 0.91 0.79 0.79 0.83
CA05042125L 0.91 0.88 0.84 0.87
MD05021815R 0.85 0.83 0.78 0.82
MD05042226L 0.86 0.81 0.68 0.79
SC05030303R 0.84 0.78 0.72 0.78
All Subjects 0.87 0.82 0.76 0.82
Relative Overlap of Manual and Neural Network Segmentation
ANN Results Compared to Manual Rater
![Page 18: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/18.jpg)
ANN output & 3D physical surface scans
Example Distance Maps
![Page 19: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/19.jpg)
ANN Validation
Subject ID Proximal Phalanx (mm)
Middle Phalanx (mm)
Distal Phalanx (mm)
Finger Average (mm)
CA05042125L 0.23 0.12 0.17 0.17
MD05021815R 0.18 0.16 0.16 0.17
MD05042226L 0.35 0.27 0.97 0.53
SC05030303R 0.26 0.17 0.20 0.21
Bone Average 0.26 0.18 0.38
ANN output & 3D physical surface scans
![Page 20: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/20.jpg)
Conclusion
• Neural networks provide a promising automated segmentation tool for identifying bony regions of interest
• Output was compared to both manual raters and 3D surface scanning– Error was less than the size of 1 voxel
• Use of 3D surface scanning provides a means to have a true gold standard for evaluation of automated segmentation algorithms
![Page 21: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/21.jpg)
Acknowledgements
• Grant funding– R21 (EB001501)– R01 (EB005973)
• Stephanie Powell, Nicole Kallemeyn, Nicole DeVries, Esther Gassman
![Page 22: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/22.jpg)
Validation
• Aim 3: Model Validation: Cadaveric specimens will be used (i) to generate three-dimensional surface scans with which surfaces defined both manually and via the automated neural network will be compared and (ii) to directly validate the computational models developed via the automated meshing algorithms.
![Page 23: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/23.jpg)
Validation
• True “gold-standard” often very difficult to achieve– Brain imaging often have to live with manual raters– Established guidelines based on anatomical experts
• Are there better “gold-standards” for other regions of the body?
![Page 24: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/24.jpg)
Orthopaedic Imaging
• Ideas developed out of goal to automate the definition of bony regions of interest.
• How can we validate these automated tools?
• Orthopaedic applications: Would it be possible to dissect cadaveric specimens?– Use bony specimen as the “gold-standard”
![Page 25: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/25.jpg)
Surface Comparison
Average Distance (mm)
Maximum Distance (mm)
Proximal 0.27 1.7
Middle 0.27 1.3
Distal 0.30 1.4
• Physical surface scan (white)
• Manually segmented surface (blue)
Proximal Middle Distal
![Page 26: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/26.jpg)
Manual surface definitions with various degrees of smoothing
(a) Unsmoothed, (b) Image-based smoothing, & (c) Laplacian surface-based
smoothing.
a b c
![Page 27: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/27.jpg)
0
25
50
75
100
CA05042125L MD05010306R MD05021815R MD05042226L SC05030303R
Specimen
Ave
rag
e P
erce
nta
ge
unsmoothed image-based Laplacian
![Page 28: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/28.jpg)
MD05021815R
0
20
40
60
80
100
proximal middle distal
Phalanx Bone
Per
cen
tag
e
unsmoothed image-based Laplacian
MD05042226L
0
20
40
60
80
100
proximal middle distal
Phalanx Bone
Per
cen
tag
e
![Page 29: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/29.jpg)
Average Euclidean distance and standard deviation between the manually traced unsmoothed surfaces and the physical surface scans.
Finger ID Proximal Phalanx (mm)
Middle phalanx (mm)
Distal phalanx (mm)
Finger Average (mm)
CA05042125L 0.19 (0.43) 0.12 (0.16) 0.15 (0.19) 0.15
MD05010306R 0.04 (0.08) 0.06 (0.09) 0.05 (0.07) 0.05
MD05021815R 0.05 (0.07) 0.05 (0.07) 0.06 (0.08) 0.05
MD05042226L 0.21 (0.32) 0.24 (0.35) 0.21 (0.29) 0.22
SC05030303R 0.20 (0.35) 0.10 (0.15) 0.10 (0.18) 0.13
Bone Average 0.14 0.11 0.11
![Page 30: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/30.jpg)
Average Euclidean distance and standard deviation between the surfaces generated via image-based smoothing and the physical surface scans.
Finger ID Proximal Phalanx (mm)
Middle phalanx (mm)
Distal phalanx (mm)
Finger Average (mm)
CA05042125L 0.32 (0.31) 0.27 (0.19) 0.32 (0.25) 0.31
MD05010306R 0.16 (0.15) 0.24 (0.19) 0.28 (0.23) 0.22
MD05021815R 0.14 (0.10) 0.18 (0.14) 0.19 (0.14) 0.17
MD05042226L 0.38 (0.30) 0.41 (0.35) 0.42 (0.37) 0.40
SC05030303R 0.36 (0.31) 0.24 (0.20) 0.27 (0.26) 0.21
Bone Average 0.27 0.27 0.30
![Page 31: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/31.jpg)
Average Euclidean distance and standard deviations between the surfaces generated via Laplacian surface-based smoothing and the physical surfaces.
Finger ID Proximal Phalanx (mm)
Middle phalanx (mm)
Distal phalanx (mm)
Finger Average (mm)
CA05042125L 0.18 (0.48) 0.10 (0.15) 0.21 (0.39) 0.16
MD05010306R 0.09 (0.15) 0.17 (0.22) 0.25 (0.37) 0.17
MD05021815R 0.03 (0.06) 0.08 (0.13) 0.06 (0.09) 0.06
MD05042226L 0.22 (0.37) 0.32 (0.54) 0.37 (0.67) 0.30
SC05030303R 0.26 (0.45) 0.11 (0.17) 0.21 (0.41) 0.19
Bone Average 0.16 0.16 0.22
Average Euclidean distance and standard deviations between the surfaces generated via Laplacian surface-based smoothing and the physical surfaces.
![Page 32: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/32.jpg)
a b c
![Page 33: Neural Network Segmentation and Validation](https://reader033.fdocuments.us/reader033/viewer/2022052510/56815c2c550346895dca082d/html5/thumbnails/33.jpg)
Neural Networks
• A computing paradigm that is designed to modeled how the brain processes data
• The network consists of several interconnected neurons that process that the input information through and activation function to form an output
• What information can be used to segment regions of interest from images