Image Processing and Cartography with the NASA Vision Workbench
-
Upload
matt-hancher -
Category
Technology
-
view
2.038 -
download
0
Transcript of Image Processing and Cartography with the NASA Vision Workbench
![Page 1: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/1.jpg)
Intelligent Systems Division NASA Ames Research Center
Image Processing and Cartography with the NASA
Vision Workbench
Matthew D. HancherIntelligent Systems Division
NASA Ames Research CenterSeptember 26, 2007
![Page 2: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/2.jpg)
Intelligent Systems Division NASA Ames Research Center
Talk Overview
• Who We Are
• Introduction to the Vision Workbench
• Example Applications
• FOSS and NASA
![Page 3: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/3.jpg)
Intelligent Systems Division NASA Ames Research Center
NASA Ames Research Center
• NASA’s Silicon Valley research center
• Small spacecraft
• Supercomputers
• Intelligent Systems
• Human Factors
• Thermal protection systems
• Aeronautics
• Astrobiology
![Page 4: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/4.jpg)
Intelligent Systems Division NASA Ames Research Center
GIS & Imaging at Ames
NASA World Wind MASTER(MODIS/ASTER simulator)
NASA/GooglePlanetary Content
Western StatesFire Monitoring Mission
![Page 5: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/5.jpg)
Intelligent Systems Division NASA Ames Research Center
IRG & ACES
Intelligent RoboticsGroup
Adaptive Control &Evolvable Systems Group
![Page 6: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/6.jpg)
Intelligent Systems Division NASA Ames Research Center
Intro to the Vision Workbench
![Page 7: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/7.jpg)
Intelligent Systems Division NASA Ames Research Center
NASA Vision Workbench
• Open-source image processing and machine vision library in C++
• Developed as a foundation for unifying raster image processing work at NASA Ames
• A “second-generation” C++ image processing library, drawing on lessons learned by VXL, GIL, VIGRA, etc.
• Designed for easy, expressive coding of efficient image processing algorithms
![Page 8: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/8.jpg)
Intelligent Systems Division NASA Ames Research Center
Open-Source VW Modules
• Core: Low-level types & platform support
• Math: General-purpose mathematical tools
• Image: Basic image operations, filters, etc.
• FileIO: Simple, flexible image file IO layer
• Camera: Camera models & related tools
• Cartography: Geospatial image manipulation
• Mosaic: Image mosaicing & multi-band blending
• HDR: High-dynamic-range imaging
(Open source as of now)
VW “Foundation”
Modules
![Page 9: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/9.jpg)
Intelligent Systems Division NASA Ames Research Center
VW Modules Underway
• InterestPoint: Interest point detection & matching
• Stereo: Stereo correlation & 3D reconstruction
• Python: Python bindings to many VW capabilities
• GPU: GPU-accelerated image operations
• Texture: Texture analysis & matching
• Display: Image display and user interaction
(The first four to be released later this year)
![Page 10: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/10.jpg)
Intelligent Systems Division NASA Ames Research Center
Design Goals & Approach
• A simple, clean API for easy hacking
• Simple syntax: Write what you mean!
• Easy to manipulate arbitrarily large images
• Automatic memory management
• Generates high-performance code
• Optimized processing via lazy evaluation
• Function inlining via “generic” (template-based) C++ style
![Page 11: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/11.jpg)
Intelligent Systems Division NASA Ames Research Center
API Philosophy
• Simple, natural, mathematical, expressive
• Treat images as first-class mathematical data types whenever possible
• Example: IIR filtering for background subtraction
background += alpha * ( image - background );
• Direct, intuitive function calls
• Example: A Gaussian smoothing filter
result = gaussian_filter( image, 3.0 );
![Page 12: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/12.jpg)
Intelligent Systems Division NASA Ames Research Center
Image Module Basics
![Page 13: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/13.jpg)
Intelligent Systems Division NASA Ames Research Center
Under the Hood: Image Views
• The core “image view” concept:• Can be evaluated at a location to return a pixel value
• Has a width and height in pixels
• Cannonical example: the ImageView class•ImageView<PixelRGB<uint8> > image(1024,768);
• Data processing represented as views
•image2 = gaussian_filter(image1, 3.0);
• Lazy container for arbitrary views
•ImageViewRef<PixelRGB<uint8> > image3 = gaussian_filter(image1, 3.0);
![Page 14: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/14.jpg)
Intelligent Systems Division NASA Ames Research Center
Image Views II
• Eliminates unnecessary temporaries• background += alpha * ( image - background );
• Supports procedurally generated images•image2 = fixed_grid(10,10,white,black,1024,768);
• Allows greater control over processing•image2 = block_rasterize( gaussian_filter(image1, 3.0) );
• Views of images on disk
•DiskImageView<PixelRGB<uint8> > disk_image(filename);
![Page 15: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/15.jpg)
Intelligent Systems Division NASA Ames Research Center
Applications & Modules
![Page 16: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/16.jpg)
Intelligent Systems Division NASA Ames Research Center
GigaPan Panorama Stitcher
(As featured in the GigaPan layer in Google Earth.)
![Page 17: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/17.jpg)
Intelligent Systems Division NASA Ames Research Center
Mosaic Module
• ImageComposite
• Composite an arbitrary number of arbitrarily large images
• It’s “just another image view”
• Supports multi-band blending for seamless composites
• QuadTreeGenerator
• Generates a tiled pyramid representation of an arbitrary image view on disk
• Great for building e.g. KML superoverlays or TMS maps
![Page 18: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/18.jpg)
Intelligent Systems Division NASA Ames Research Center
Cartographic Reprojection
(As seen in the newly updated Google Moon.)
![Page 19: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/19.jpg)
Intelligent Systems Division NASA Ames Research Center
Cartography Module
• GeoReference• Uses PROJ.4 for standard projections, GDAL to read/write
• GeoTransform• Reprojects image data between GeoReferences
• Makes “just another image view”
• OrthoImageView• Ortho-rectifies an aerial or satellite image against an
arbitrary DEM (in conjunction with the Camera module).
• Also “just another image view”
![Page 20: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/20.jpg)
Intelligent Systems Division NASA Ames Research Center
Automated Image Alignment
• Problem: Given two images, find and align the overlap region.
![Page 21: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/21.jpg)
Intelligent Systems Division NASA Ames Research Center
Image Alignment w/ Interest Points
Images to be alignedLocate interest points in first imageLocate interest points in second imagePoint correspondences determine image alignment
![Page 22: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/22.jpg)
Intelligent Systems Division NASA Ames Research Center
Interest Point Module
• Interest point detectors, descriptors, and matching
ScaledInterestPointDetector<LoGInterest> detector;InterestPointList ip1 = interest_points( image1, detector );InterestPointList ip2 = interest_points( image2, detector );
PatchDescriptor descriptor;compute_descriptors( image1, ip1, descriptor );compute_descriptors( image2, ip2, descriptor );
DefaultMatcher matcher(threshold);InterestPointList matched1, matched2;matcher.match( ip1, ip2, matched1, matched2 );
Matrix2x2 homography = ransac( matched1, matched2, SimilarityFittingFunctor(), InterestPointErrorMetric() );
![Page 23: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/23.jpg)
Intelligent Systems Division NASA Ames Research Center
The Ames Stereo PipelineFast, high quality, automated stereogrammetric surface reconstruction originally developed for
Mars Pathfinder science operations
Disparity
Now a Vision Workbench application.
![Page 24: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/24.jpg)
Intelligent Systems Division NASA Ames Research Center
Primary Image Secondary Image
RegistrationEphemeris or
Automated Interest Points
Fast Stereo Correlation
Outlier Rejection / Hole Filling / Smoothing
Disparity Map
Camera Model (e.g. Linear Pushbroom)
Mesh Generation
Point Cloud/DTM
3D Mesh
Mask / Sign of Laplacian of Gaussian
The Ames Stereo Pipeline
Surprise: It’s all just Vision Workbench image views!
![Page 25: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/25.jpg)
Intelligent Systems Division NASA Ames Research Center
Mars Stereo: MOC NA
MGS MOC-Narrow Angle• Malin Space Science Systems• Altitude: 388.4 km (typical) • Line Scan Camera: 2048 pixels• Focal length: 3.437m• Resolution: 1.5-12m / pixel• FOV: 0.5 deg
![Page 26: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/26.jpg)
Intelligent Systems Division NASA Ames Research Center
NE Terra Meridiani
Upper Left: This DTM was generated from MOC images E04-01109 and M20-01357 (2.38°N, 6.40°E). The contour lines (20m spacing) overlay an ortho-image generated from the 3D terrain model. Lower Right: An oblique view of the corresponding VRML model.
!!"""#$
!!"""$
!%""#$
!%""#$
!%""#$
![Page 27: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/27.jpg)
Intelligent Systems Division NASA Ames Research Center
Preliminary MOLA Comparison
Scanline Capture Time (s)
Elev
atio
n at
bor
esig
ht p
ixel
(m
)
![Page 28: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/28.jpg)
Intelligent Systems Division NASA Ames Research Center
Lunar Stereo: Apollo Orbiter Cameras
ITEK Panoramic Camera• Focal length: 610 mm (24”)• Optical bar camera• Apollo 15,16,17 Scientific
Instrument Module (SIM)• Film image: 1.149 x 0.1149 m• Resolution: 108-135 lines/mm
![Page 29: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/29.jpg)
Intelligent Systems Division NASA Ames Research Center
Apollo 17 Landing Site
Top: Stereo reconstruction
Right: Handheld photo taken by an orbiting Apollo 17 astronaut
![Page 30: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/30.jpg)
Intelligent Systems Division NASA Ames Research Center
Public Outreach: Hayden Planetarium
![Page 31: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/31.jpg)
Intelligent Systems Division NASA Ames Research Center
Public Outreach: Hayden Planetarium
![Page 32: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/32.jpg)
Intelligent Systems Division NASA Ames Research Center
Application: Image Matching
• Problem: Given an image, find others like it.
Example database: Apollo Metric Camera images
![Page 33: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/33.jpg)
Intelligent Systems Division NASA Ames Research Center
Texture-Based Image Matching
Texture bank filtering (Gaussian 1st derivative and LOG)
Grouping to remove orientationEnergy in a window
E-M Gaussian mixture modelIterative tryouts, MDL
Max vote
GroupingMean energy in segment
Euclidian distance
Summarization
Post-processing
Output Representation
Filtering
Model image
Segmentation
Vector Comparison
Matched image
![Page 34: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/34.jpg)
Intelligent Systems Division NASA Ames Research Center
Image Matching: Results
![Page 35: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/35.jpg)
Intelligent Systems Division NASA Ames Research Center
FOSS and NASA
![Page 36: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/36.jpg)
Intelligent Systems Division NASA Ames Research Center
The NOSA
• The NASA Open Source Agreement, an OSI-approved non-viral open source license
• Intended to protect users from contributor patent licensing issues.
• Yes, we know: The current version (1.3) has several well-known peculiarities.
![Page 37: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/37.jpg)
Intelligent Systems Division NASA Ames Research Center
U.S. Contractor Rights
• The University and Small Business Patent Procedures Act of 1980, a.k.a. “Bayh-Dole”.
• A university, small business, or non-profit can claim patent ownership of a federally-funded invention before the government.
• The government must actively promote and attempt to commercialize the invention.
• Severely complicates open-source initiatives within the government that involve universities, small businesses, or non-profits.
![Page 38: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/38.jpg)
Intelligent Systems Division NASA Ames Research Center
The Open Source Process
• Open-source approval stages include:
• Invention disclosure
• Copyright assignment (all parties)
• Legal review (copyright & patent issues)
• Export control review (e.g. ITAR)
• Computer security review
• more....
![Page 39: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/39.jpg)
Intelligent Systems Division NASA Ames Research Center
Signs of Improvement
• The old model: (e.g. VW 1.0)• Seek approvals after code completion
• Long, slow, high-latency release cycle
• The new model: ?? (e.g. WV 2.0 ??)• Seek periodic approval for upcoming development
• Allows regular updates within prescribed bounds
• On the horizon: ??• User contribution process ?
• Publicly-accessible subversion repository ???
![Page 40: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/40.jpg)
Intelligent Systems Division NASA Ames Research Center
Free and Open Data
• Free and open data has received much less attention than free and open software.
• The National Aeronautics & Space Act:
• The Administration, in order to carry out the purpose of this Act, shall... provide for the widest practicable and appropriate dissemination of information concerning its activities and the results thereof.
• Alas, NASA does not own much of what is often imagined to be “NASA data”.
![Page 41: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/41.jpg)
Intelligent Systems Division NASA Ames Research Center
Outreach: Google Earth
MODIS CoveragesAstronaut Photography
• Make more datasets publicly available as KML (and soon WMS) for mash-ups.
• Increase the visibility of existing public repositories of NASA data and imagery.
![Page 42: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/42.jpg)
Intelligent Systems Division NASA Ames Research Center
Outreach: Google Moon
Data coming soon via KML and WMS from NASA.
![Page 43: Image Processing and Cartography with the NASA Vision Workbench](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d4c4f5bb61eb307c8b4581/html5/thumbnails/43.jpg)
Intelligent Systems Division NASA Ames Research Center
Obtaining the Vision Workbench
• VW version 1.0.1 available now.
• VW version 2.0 coming this fall!
• To contact me:
http://ti.arc.nasa.gov/visionworkbench/