20 April 2023
Chair for Computer Aided Medical Procedures & Augmented Reality
Department of Computer Science | Technische Universität München
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Automatic Feature Generation for Endoscopic Image Classification
Ulrich Klank
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 2
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Endoscopic Image Classification
Synchronization of surgeries by phases is essential for Workflow Project1
Analyzing endoscopic images supports the classification of the current state into a phase.
1 A. Ahmadi, T. Sielhorst, R. Stauder, M. Horn, H. Feussner, N. Navab -Recovery of surgical workflow without explicit models
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 3
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Task
How can these images be classified into surgical phases?
Or if they can’t be classified well enough, which features can be added to the rest of the information?
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 4
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Definition of a Feature
Feature as representation of discriminative information Numerical vector calculated out of the
image Reduction of the dimension
Features Basic: Gradient/Color/Motion Histogram Integral image1
Phase concruency2
Gradient Histogram
1 Viola, Jones- Robust Real-time Object Detection
2 Kovesi - Image Features from Phase Congruency
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 5
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Feature Space
With a limit to algorithms with an output smaller or equal than the input we get a number of injective function about:
depending on the dimension of the input vector d and the number of different values c which are possible in the input.
Exploration of this space with Genetic Programming1 A method to explore a high dimensional space Fitness function influences the resulting features
1 Feature Extraction from Multiple Data Sources Using Genetic Programming - John J. Szymanski,* Steven P. Brumby, Paul Pope, Damian Eads, Diana Esch-Mosher,Mark Galassi, Neal R. Harvey, Hersey D.W. McCulloch, Simon J. Perkins, Reid Porter,James Theiler, A. Cody Young, Jeffrey J. Bloch and Nancy David
(cd)cd( cd¡ 1)
2
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 6
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Feature Generation by Genetic Programming12
Mutation
EvaluationCode
1 Wolfgang Banzhaf, Peter Nordin, Robert E. Keller, Frank D. Francone - Genetic Programming: An Introduction
2 Koza - Genetic Programming
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 7
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Implementation Steps
Definition of a programming language (more) Designed for recombination and mutation
Designing a Framework to run programs Translation of basic features in the new language Evaluation of the fitness of programs Recombination and Mutation
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 8
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
An Example as Proof of Concept
Implementation of a color histogram in the new language
Test on an easy sample, 6 images(10*10*RGB, jpeg) in 3 classes:
Dimension of input here (cd): 76.800
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 9
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
The Fitness Function is the core of the system
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 10
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
Comparison of several Programs Pi
P1P1 P2
P2…
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 11
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
A random selection of the labeled input Ik
several from of each class
P1P1 P2
P2 …
I1 I2 I3
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 12
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
Execution of the program with every selected input Ik
As result there are output vectors:
P1P1 P2
P2 …
I1 I2 I3
o11;o12 ¢¢¢o23
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 13
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
Compare each pair of output vectors of one Program, by the distance
P1P1 P2
P2 …
I1 I2 I3
dik;l =
P lenm=0
(oi k [m]¡ oi l [m])2
(joi k [m]j+joi l [m]j)2
o11;o12 ¢¢¢o23
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 14
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the Example
Select the maximum of these distances, only considering pairs of different classes:
P1P1 P2
P2 …
I1 I2 I3
o11;o12 ¢¢¢o23
dik;l =
P lenm=0
(oi k [m]¡ oi l [m])2
(joi k [m]j+joi l [m]j)2
Now we have two classes k, l which our feature separates
argmaxkl(dikl jk:class 6= l:class)
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 15
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evaluation of the example
Subtract the distances in images of the classes k or l
P1P1 P2
P2 …
I1 I2 I3
o11;o12 ¢¢¢o23
dik;l =
P lenm=0
(oi k [m]¡ oi l [m])2
(joi k [m]j+joi l [m]j)2 f i = dikl ¡ di
k k +dil l
2argmaxkl(di
kl jk:class 6= l:class)
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 16
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
3.1 Results in this Example
< 64 >
< 72 >
< -16 >
< 72 >
< -16 >
< -16 >
NewProgram
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 17
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Fitness in the Example:
Input Program was calculating a Color Histogram (Output length
512 in ~2000 steps), resulting fitness: ~ 1.2 % The new generated program calculated one number (in 30
steps) and got a fitness of: ~ 98.8 %
P1P1
P2P2
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 18
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Evolution Technique used in the Example
Selection of the two best Program and generation several children by these two methods:
One parental program: Mutation Adding of random commands Removing a single commands1 Duplicating an existing command Switching commands
Two parental programs: Breeding Mixing both, preferring blocks of a specific size Adding random commands in the gaps between
P1P1
P11P11
P1P1 P2
P2
P12P12
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 19
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Conclusion
The concept seems to work Adaptation to workflow project’s data can start
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 20
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Steps to adapt it to the Workflow project
Adapt fitness function (more) Change Genetic rules (more) Optimize the framework Generate C code out of the intermediate
language
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 21
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Any Questions?
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 22
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Appendix
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 23
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Planned Improvements in the Genetic rules
Probability to survive should depend on the fitness of all other existing programs
Even a program with a low fitness should have a small change to survive
Also the reproduction should not be restricted to the best program
Back
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 24
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Fitness
Actual fitness expresses: Linear separation of two classes
Further improvements: Static Analysis of Code1
Semantic Checks to prevent running in infinite loops to prevent the execution of programs not using the input …
Other Classifiers to evaluate the output
1 Colin G. Johnson-Deriving genetic programming fitness properties by static analysis
Back
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 25
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Specification of the language
Assembler style with some high level functions
1 Opcode and 3 Arguments for one command
High level examples: Reading of the input and its dimensions For loops
Adding of more functions possible
Back
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 26
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Scheme for the programming language
OPCode 1.Parameter
2. Parameter
3. Parameter
010
For
counter to stack
0
From
Constant
100
To
Constant
65
End of the loop
Constant
123
Read pixel
Value to stack
-1
x
From stack
20
y
Constant
…
…
CAMP | Department of Computer Science | Technische Universität München | 20 April 2023 27
Chair for Computer Aided Medical Procedures & Augmented Reality | wwwnavab.cs.tum.edu
Generated Program (without nops)
INPUTDIMENSION 0 0 4595026
SAVE 0 -1 0
INPUTDIMENSION 1 0 0
SAVE 23367 -1 0
FOR 0 -1 37
LOAD 4 0 0
LOAD 1 0 0
SAVE 5 -1 0
LOAD 5 0 0
POP 0 0 -1
READINPUT -1 -1 -1
LOAD 5 0 -1
POP 0 0 -1
PUSH 64 0 -1
SAVE 3 -1 0
PUSH 8 0 0
MULTIPLY 0 0 0
LOAD 3 0 0
PLUS 0 12604 0
PLUS 0 6457 0
PLUS 31190 0 0
JUMP 55 0 0
DIVIDE 0 0 0
DIVIDE 0 0 0
PUSH 32 0 0
LOAD 1 -1 0
LOAD 2 0 0
RETURN 0 0 0
JUMP 0 0 0
FOR 8 520 58
Top Related