The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.
-
Upload
osborne-ford -
Category
Documents
-
view
215 -
download
1
Transcript of The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.
![Page 1: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/1.jpg)
The Implementation of Markerless Image-based 3D Features Tracking System
Lu Zhang
Feb. 15, 2005
![Page 2: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/2.jpg)
Motivations
Objective Find more efficient algorithms to implement 3D volume tracking
based on 2D Image sequences.
Problems in this topic 1. Huge datasets For only one data file: 128*128*128 2. Computation time
Applications 1. On sensors 2. On robotics
![Page 3: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/3.jpg)
Outline
Previous work Flowchart of the system Algorithms
Current work Improved algorithms Comparisons
Future work Problems unsolved How to enhance computation speed
![Page 4: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/4.jpg)
Previous Work
The original image-based 2D dataset :
Size: 512*512*40*(R, G, B)
Flowchart and Modulus
Input imagesSegmentation
Feature extraction
Classification
Graph building
Basic features
classes
Directed acyclic graph
![Page 5: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/5.jpg)
Previous Work - Algorithms Modulus1: Segmentations
Global Thresholding:
Problems: One threshold to all image sequences. Iterative region growing method [1]
After applying this method to segmented image sequences:
VS
![Page 6: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/6.jpg)
Previous Work-Feature Extraction
Output from Feature extraction module
viewID mx my areas labeling timeID label.
Modulus2: Feature Extraction
After gaining region information from segmentation stage, we can browse each region to find basic features:
Areas – The count of all pixels in the region.Center of Gravity –The center of all points in one region.
![Page 7: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/7.jpg)
Classification /Feature tracking
Modulus3: Classification One Assumption
Time between successive data sets is small: we can assume the difference between a pair of views should not vary too much.
Euclidean Distance Classifier
Aeuc xAxd ,
![Page 8: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/8.jpg)
Current Work-Improvement Modulus 1: Segmentation: Optimal Thresholding: Isodata algorithm
Segment images into two parts using a starting threshold value.
Calculate the mean (mf,0) of the foreground pixels and the mean (mb,0) of background pixels.
A new threshold value is now computed as the average of these two sample means.
The process is repeated, based upon the new threshold, until the threshold value does not change any more.
![Page 9: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/9.jpg)
Previous Work - Algorithms
Modulus1: Segmentations Region growing:
Purpose: Trying to separate overlapped objects
Algorithms: Region growing based on Marr-Hildreth and sobel edge detectors
![Page 10: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/10.jpg)
Current Work-Features extraction
Feature Extraction Diameter - Diameter is the distance between two points on
the boundary of the region whose mutual distance is the maximum.
Major Axis of The Region – the major axis of the region is the line which minimizes:
These two features are relatively robust, and the second feature: major axis can help detect the reflection part on objects.
n
iid
1
22d
![Page 11: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/11.jpg)
Current Work-Features extraction
Feature Extraction Compute major axis
PCA Diameter
1. Rotate the X-Y coordinate to let the new X-coordinate is the major axis
2. Divide the 2D plane into four regions, find the furthest points on each region
3. Calculate the Euclidean distance
)cos()sin(
)sin()cos(Q
![Page 12: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/12.jpg)
Current Work-Features extraction
Experiment results from diameter detector
![Page 13: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/13.jpg)
Current Work-Features extraction
Experiment results of from feature extraction modulus:
TimeID ViewID Mx My R G B areas diameter angle label
![Page 14: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/14.jpg)
Current Work-Feature Extraction
Modulus2: Feature Extraction Problems solving:
Reflections: According to the experiment result on the right:
to some big objects, their reflections which come from the distance transformation when we pre-projected 3D objects onto 2D image plane
are distracted as different objects. Algorithms: Using the property of major axis: because they belong to the same object, their major axis should parallel or at least have similar angles to each other
![Page 15: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/15.jpg)
Current work-Classification /Feature tracking
Classification methods
Euclidean Distance Classifier
Evolution in time-varying images
There are five different changes of regions between a pair of views.
Continuation: one feature continues from dataset at t1 to the next dataset at t2
Creation: new feature appear in t2 Dissipation: one feature weakens and becomes part of the background Bifurcation: one feature in t1 separates into two or more features in t2. Amalgamation: two or more features merge from one time step to the
next.
![Page 16: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/16.jpg)
Current Work
Output from Classification module New class to preserve the output dataset from Classification module:
class LabelTrack(). It preserve the information:1. ViewID: camera positions, we will move camera around the object in
order to restore 3D object.
2. timeID: time order, for each camera position , we will take several time- varying images
3. classID: class number after correspondence computation between a pair of images in time order
4. Label: the original region numbers before correspondence computaton
5. R, G, B: the color information for each pixel
6. Coordinate x, y: the 2D coordinate of the projection of 3D object.
7. Forward pointer: preserve the labeling information of the previous dataset
8. Backward pointer: preserve the labeling information of the next dataset
![Page 17: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/17.jpg)
Future Work-Speed Enhancement
The importance of computation time Size of mine dataset:
512*512*24*40(time orders)*N(camera positions)
In [5], the computation time for 128^3*10 is 7 minutes.
In the previous work, I use 4 minutes for 512*512*24*40.
In the current work, most I/O operations have been removed, although the computation time is around 5 minutes. Most of the time is consumed on Marr-Hildreth edge detector.
![Page 18: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/18.jpg)
REFERENCES
[1] Snyder and Cowart, “An Iterative Approach to Region Growing”, IEEE transaction on PAMI, 1983
[2] Wesley E.Snyder and Hairong Qi, “Machine Vision”, Cambridge [3] Richard O.Duda, Peter Hart, David Stork, “Pattern Classification”, Prenti
ce Hall [4] Rafael Gonzalez, Richard Woods,”Digital Image Processing”, 2nd, Prenti
ce Hall [5] D.Silver, Xin Wang, ”volume tracking”, Visualization '96. Proceedings.27
Oct.-1 Nov. 1996
![Page 19: The Implementation of Markerless Image-based 3D Features Tracking System Lu Zhang Feb. 15, 2005.](https://reader034.fdocuments.us/reader034/viewer/2022042821/56649f475503460f94c69272/html5/thumbnails/19.jpg)
Thanks
Any questions?