Integrated Field Testing of Planetary Robotics Vision...
33
Page 1 Version: 28.05.2012 [email protected] www.provisg.eu Integrated Field Testing of Planetary Robotics Vision Processing: The PRoVisG Campaign in Tenerife 2011 Gerhard Paar, Lester Waugh, Dave Barnes, Tomas Pajdla, Mark Woods, Hans-Rudolf Graf, Yang Gao, Konrad Willner, Jan-Peter Muller, Rongxing Li, Michel Maurette
Transcript of Integrated Field Testing of Planetary Robotics Vision...
Page
1
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Integrated Field Testing of Planetary Robotics Vision Processing: The
PRoVisG Campaign in Tenerife 2011 Gerhard Paar, Lester Waugh, Dave Barnes, Tomas Pajdla,
Mark Woods, Hans-Rudolf Graf, Yang Gao, Konrad Willner, Jan-Peter Muller, Rongxing Li, Michel Maurette
Page
2
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
PRoVisG • European Research Project (FP7-SPACE)
– 14 Institutions in EU & US – 45 Months – 373 Person Months
• Planetary Robotics Vision Ground Processing – Enhance Planetary Robotics Vision – Collect tools & compile integrated SW suite – Test the result – Provide relevant (controlled) test data
Page
3
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Logistics and Administration
• Objectives for the campaign were defined • Description of Locations and their specific features • Setup of a global schedule • Contacts of participants and general logistics
– accommodation, site access, personal equip. • Description of test facilities • On-site preparation
Page
4
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
PRoViP & PRoGIS Use Cases: Objectives • On-Site Verification of data & data integrity • Demonstration of full chains • Demonstration of (immediate) results to Public • Derivation of mission
issues – Operations – Science
• Hints for further data acquisition requirements – Vision Data – Reference Data
Page
5
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Logistics and Administration • Description of instruments and
components involved in the tests • Definition of operation, processing
and storage procedures • Description of communication
with non-on-site places • Breakdown of global plan into
experiments and daily plan
Page
6
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Overview: „Bridget Sensors“
AUPE Exomars PanCam Emulator: • 2 Wide Angle Cams (WAC)
with filter wheel (RGB or narrow-band)
• 360x135 degree panoramas
• one High Resolution Cam (HRC)
UCL HyperCam (HC1) • 658 x 496 pixel x 16 bit
EMCCD, C-mount lens and visible / NIR LCTF
• 10nm sampling
Ultrasonic Drill • Univ. of Glasgow (external)
CSEM Catadioptric Stereo Camera • 2 vertically stacked Omniview
cameras (distance: 400mm)
• theoretical resolution: 10mm at a distance of 1000mm
CSEM 3D Tof • SR4000 from MESA (CSEM)
• 176x144 pixels, 24 power LEDs at 850nm
• distance resolution: 4mm in a distance range of 0.8 to 5m
main vision sensor in Field Trials
Page
7
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Contributors
Page
8
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Course of the Field Trial • Following the extensive preparation the 9 day field
trial on Tenerife was held
Schedule 12th Sept - Mon - Setup 13th Sept - Tue - Shakedown 1 14th Sept - Wed - Shakedown 2 15th Sept - Thu - Trial Day 1 16th Sept - Fri - Press Day videos 17th Sept - Sat - Open Day 18th Sept - Sun - Trials Day 2 19th Sept - Mon - Trials Day 3 20th Sept - Tue - Rover packup
Page
9
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Shakedown • Final assembly of the rover and its equipment • First driving and integration tests • Fine tuning of the rover and instruments • Establish network and control connections • test remote driving • fix all occuring problems
All this prior to the first scheduled and protocolled tests to ensure maximum outcome!
Page
10
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Field Trial Day 1 • Rover needed repair – took until the late afternoon to fix all
cables and test the rover • PRoVisG team went to field with a tripod to keep testing their
equipment in the Morning • Afternoon: Rover went to Minas de San Jose and the team
managed to get through most of the envisaged imaging experiments
Page
11
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Press Day / Open Day • Several TV stations called in and reported about the field trial
during Press Day • Local people and tourist were given the opportunity to get a
close look on Bridget during the Open Day. All, especially kids enjoyed it!
• Besides the visitors; 70% of imaging plan was fullfilled
Members of European Parliament
Page
12
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Overview Experiments nr experiment realized evaluated / processed
1 Single RGB Stereo Panorama X (day 1&2) X
2 Full spectrum Stereo Panorama X (day 1&2)
3 Environmental dynamics test X
4 3D-TOF & Omniview Panorama X (day 3)
5 Omniview calibration experiment X (day 3)
6 super resolution panorama X
7 HRC APIC Mosaik
8 Visual Odometry (VO) Loop Sequence X (day 2) X
9 On-line Data Processing Demo X CTU
10 Rover External Localization X (day 2&3) X
11 Integrated Rover Localization X
12 Wide Baseline Mapping x
13 HyperCam Multispectral Mosaics X (several days) Partly. No mosaics
Page
13
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
2 Sites
Page
14
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Data acquired • AUPE PanCam was the main vision sensor for the Field Trials.
More than 7000 images were taken, the following list summarizes the acquired data sets: – At Llanos de Ucanca, 5 panoramas (two 180° pan, three 360° pan) were
acquired, partly in RGB. – At Minas de San Jose, more than 15 stereo WAC panoramas with different
angular pan & tilt range were taken, most of them in RGB. – Parts of panoramas were covered with the full WAC wavelength range. – High dynamic range sequences were taken to proof the benefit of > 12 bit
radiometric range – A trajectory of about 400 meters was covered with stereo visual odometry
sequences, with 1-2 meters of intervals. – super resolution sequences (using about ten slightly different – random –
pan-tilt values from the same scene) were acquired. – Also the HRC from AUPE was used to ensure the existence of a realistic test
data set for ExoMars PanCam processing.
Page
15
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Data Acquired III • Hypercam
– Hypercam was applied on four different sites, all of them also covered with AUPE.
• Onmiview – At Minas de San Jose an area of about 20 * 20 meters was
traversed twice in a circular path. The Omniview Camera was operated in video mode and captured with about 10 Hz.
Page
16
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 4: 3D-TOF & Omniview video
Page
17
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 1: Single RGB Stereo Panorama – PRoVisG Summer School
• Summer School in Berlin was commenced in parallel to the Field Trial
• Students attended introductory lectures before they applied the PRoVIP software
• The Summer School got first hands information on the progress of the Field Trials during a Skype communication session
• Field Trial provided the Summer School with PanCam images that could successfully be processed on the fly to a 3D panorama scene
Page
18
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 10: Rover External Localization • Aim:
– determine 3d rover positions / trajectories from an external stereo camera setup and known 3d target positions
• Instrument Resources: – 2 external cameras (cam1, cam2 – cheap
Ricoh digicams with 30sec interval exposure)
– simple tripods
Page
19
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 10: Rover External Localization • Other Resources:
- min 1 target sphere on observed rover - several targets with known 3d target
coordinates (local photogrammetric network) - optional: for global context, GPS-measured
landmarks are manually identified in the images for geo reference
Page
20
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 10: Rover External Localization • Experiment Description:
– cameras are simultaneously triggered, acquire the whole site during all day from two positions about 20m base length
– camera positions manually adapted to always ensure the rover and a sufficient number of targets being inside the field of view
Page
21
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 10: Rover External Localization • Results (2):
– rendered video of corresponding stereo images (day 5, Minas, 18.09.2011)
Page
22
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
2nd Means of Global Orientation
Page
23
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
• Results – global positioning: – global coordinates for all rover positions available
local coos global coos img pair id …
…
Exp. 10: Rover External Localization
Page
24
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Aspects of Testing in the Public
Page
25
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 8 Visual Odometry Results (Minas4)
1. SURF parameters: Threshold = 0.003, Sampling Step = 3, Similarity Ratio = 0.85
2. Outliers detection method: Homography
3. Backprojection distance = 1.9 pixel
Page
26
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Structure from Motion (Round the Rock)
Page
27
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Hypercam – to – AUPE Coregistration • Hypothesis by cross-calibration (ims & fulcra)
• SURF matches
Page
28
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
HC1 Superimposition onto WACL/R Ortho
Page
29
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 1 3D from Minas 6 Panorama
Page
30
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Exp. 1: Minas6 Panorama
Page
31
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Pack & Clean Up • Disassembling and packing of the rover
and instruments • Test sites need to be left as found as it is
located in a national park area. Cleaning includes the removal of all tracks of the rover.
• Post field trial inspection was successful!
Lessons Learned • Unexpected things happen, be prepared. • Backup plans are essential to keep on
going even with obstacles in the way. • With time field trials become more
effective as the daily routine establishes.
Page
32
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Experiments / Planning Compliance
Page
33
Vers
ion:
28.
05.2
012
Ger
hard
.paa
r@jo
anne
um.a
t
www.provisg.eu
Further Data Exploitation • See the Blog on
http://sites.google.com/site/bridgetrover/ • Localized / Calibrated image sets will be made
available in PDS format • Further Data Processing by PRoVisG
Participants – HDR – Super Resolution – Site Fusion – ….
The End
