1 Toward Autonomous Kilometer-Scale Site Survey by Planetary Rovers David R. Thompson Robotics Institute Carnegie Mellon University NASA ASTEP NNG0-4GB66G 2 Agenda Autonomous Site Survey Samples to environments Experimental design Component technologies Feature detection Spatial models Adaptive collection/return Future challenges 3 1m 10m 10 2 m 10 3 m 10 4 m ? 2009 Optimal daily range single site multiple-site over-the-horizon 4 Exploration bottlenecks / challenges Sparse samples Cannot target features in advance Limited bandwidth Labor-intensive data analysis 5 Artificial geologic intelligence? Recognize and classify interesting features Acquire data autonomously Prioritize data products for downlink Adapt to trends and anomalies 6 Previous work Marsokhod Rover Tests (Gulick et al., 2001) Antarctic Meteorite Search (Pedersen et al., 2001) Single Cycle Instrument Placement (Pedersen et al., 2004, Madison 2006, etc.) OASIS, WATCH systems (Castao et al., 2005, 2006, 2007, etc.) EO-1 Agent, Sensorweb (Chien et al., 2005, etc.) 7 Image Courtesy NASA / JPL-Caltech / Cornell 8 Science value is context sensitive Courtesy USGS Courtesy NASA 9 An experimental design approach 1. Define geologic parameters of interest 2. Model correlations between parameters and data 3. Seek data that minimizes expected posterior entropy P( ) P( |Data) H() = - P() log P() d 10 Spatial inference Cross-sensor inference ? ? Statistical models exploit correlations 11 Agenda Autonomous site survey Motivation An experimental design approach Component technologies Feature detection Spatial models Adaptive collection/return Future challenges 12 Amboy Crater, Mojave Desert, CA feature detection data model action selection 13 z 14 1. Feature detection / classification 2. Spatial model (map) 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Remote sensing Component technologies 15 Remote sensing multispectral pixel intensities feature detection data model action selection 16 Rock detection feature detection data model action selection a b 17 Rock detection feature detection data model action selection slow and accurate quick and dirty 18 Rock detection: quick and dirty Filter Cascade A (Left-lit) detected rocks input image feature detection data model action selection max Filter Cascade B (Right-lit) 19 Rock detection: quick and dirty feature detection data model action selection 20 Reflectance Spectroscopy feature detection data model action selection 21 Spectrum collection procedure 1.Calibrate spectrometer 2.Kinematic pointing 3.Visual servoing: 1.Detect rocks in image 2.Match across images with SIFT (Lowe 2001) 3.Stereo depth -> desired pixel (u,v) 4.Pan-tilt servo update feature detection data model action selection 22 Amboy Crater spectrometer tests feature detection data model action selection 4x40 minute traverses through a field of rocks Rover constructed a map of unique rocks and associated spectra 23 Amboy Crater spectrometer tests feature detection data model action selection 24 Amboy Crater spectrometer tests feature detection data model action selection Targeted spectroscopy averaged unique rocks per run Fixed pointing never found isolated rocks Performance was sensitive to lighting 50m 25 Classification via unsupervised learning feature detection data model action selection wavelength (nm) reflectance basalt sediment 26 1. Feature detection / classification 2. Spatial model (map) 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Orbital data Component technologies 27 Spatial models: Gaussian processes feature detection data model action selection Solves a regression problem: [location, remote sensing] class Learns the appropriate degree of spatial and cross-sensor correlation C(x i,x j ) exp {- d w d |x i d - x j d |} spatial / cross-sensor structure 28 Amboy Crater tests feature detection data model action selection Built maps of eroded basalt mounds Over 100 distinct trials 10 to 60 minutes 100m to 1.0 km Geologic class from fixed-angle spectroscopy 29 Navigation sequence 30 Inference result 31 Amboy Crater tests feature detection data model action selection 32 1. Feature detection / classification 2. Spatial model (map) 3. Action selection Surface images, spectra Navigation, data collection Selective downlink Orbital data Component technologies 33 Corridor exploration scenario Restricted planning task Robot moves forward to reach end-of-day goal Periodic sampling ~500m DbDb DaDa ~100m 34 Informative path planning feature detection data model action selection Recursive path planning (Chackuri / Pal 2005) Predict entropy of future observations Maximize expected information gain Respect time and resource constraints 35 Informative path planning 36 Recovery from Navigation Error 37 collection returned data collected data environment downlink min H( environment | returned data ) feature detection data model action selection min H( environment | returned data ) 38 min X,X' H(|X') collection,, downlink XX' environment collected datareturned data feature detection data model action selection min X,X' H(|X') 39 Data Collection Maximum Entropy Sampling (Sebastiani and Wynn, 2000) Data Return Combinatorial Optimization collection,, downlink XX' environment collected datareturned data feature detection data model action selection Objective Function min X,X' H(|X') -H(,X) + H(X|X') 40 Selective data return feature detection data model action selection 41 Agenda Autonomous site survey Motivation An experimental design approach Component technologies Feature detection Spatial models Adaptive collection/return Future challenges 42 Short term: Analyze Amboy results Integrate targeted spectroscopy and mapping Long term: better feature analysis and image descriptors more sophisticated models free scheduling of science activities balance navigability and science objectives more application domains Future Challenges 43 Take-home messages Science value is context-sensitive Remote exploration as adaptive experimental design Progress toward autonomous kilometer- scale site survey 44 Thanks! David Wettergreen, Francisco Caldern, Dom Jonak, Ron Greeley, Shelby Cave, Phil Christensen, Steve Chien, Steven Flores, James Teza, Jeff Schneider, Heather Dunlop 45 Rock detection: slow and accurate feature detection data model action selection sun angle superpixels high-scoring solution segmentation search texton map initial segmentation 46 Rock detection: slow and accurate feature detection data model action selection Classification accuracy by feature Shape Shading Texture 93.8% 80.5% 94.7% 95.6% 91.2% 97.3% 91.2% None 53.1% 47 Rock detection: slow and accurate feature detection data model action selection 48 Classification via unsupervised learning , 0.4, feature detection data model action selection 49 feature detection data model action selection Principal Component Analysis