Potential Scaling Effects for Asynchronous Video in Multirobot Search
description
Transcript of Potential Scaling Effects for Asynchronous Video in Multirobot Search
![Page 1: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/1.jpg)
Potential Scaling Potential Scaling Effects for Effects for
Asynchronous Video Asynchronous Video in Multirobot Searchin Multirobot Search
Prasanna VelagapudiPrasanna Velagapudi11, Huadong Wang, Huadong Wang22, Paul , Paul ScerriScerri11, Michael Lewis, Michael Lewis22 and Katia Sycara and Katia Sycara11
11Carnegie Mellon University, USACarnegie Mellon University, USA22University of Pittsburgh, USAUniversity of Pittsburgh, USA
![Page 2: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/2.jpg)
Urban Search and Rescue Urban Search and Rescue (USAR)(USAR)
► Location and rescue Location and rescue of people in a of people in a structural collapse structural collapse
► Urban disastersUrban disasters LandslidesLandslides EarthquakesEarthquakes TerrorismTerrorism
Credit: NIST
![Page 3: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/3.jpg)
USAR RobotsUSAR Robots
► Robots can helpRobots can help Unstable voidsUnstable voids Mapping/clearingMapping/clearing
► Want them to be: Want them to be: SmallSmall CheapCheap PlentifulPlentiful
Credit: NIST
![Page 4: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/4.jpg)
Urban Search and Rescue Urban Search and Rescue (USAR)(USAR)
►Now: One operator Now: One operator one robot one robot Directly teleoperatedDirectly teleoperated Victim detection through synchronous videoVictim detection through synchronous video
►Future: One operator Future: One operator many robots many robots Manufacturing robots is easyManufacturing robots is easy Training operators is hardTraining operators is hard
►Need to scale navigation and Need to scale navigation and searchsearch
![Page 5: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/5.jpg)
Synchronous VideoSynchronous Video
►Most common form of camera Most common form of camera teleoperationteleoperation High bandwidthHigh bandwidth Low latencyLow latency
►ApplicationsApplications SurveillanceSurveillance Bomb disposalBomb disposal InspectionInspection
Credit: iRobot
![Page 6: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/6.jpg)
Synchronous VideoSynchronous Video
►Does not scale with team sizeDoes not scale with team size
![Page 7: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/7.jpg)
Synchronous VideoSynchronous Video
►Does not scale with team sizeDoes not scale with team size
![Page 8: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/8.jpg)
Synchronous VideoSynchronous Video
►Does not scale with team sizeDoes not scale with team size
![Page 9: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/9.jpg)
Asynchronous ImageryAsynchronous Imagery
► Inspired by planetary robotic solutionsInspired by planetary robotic solutions Limited bandwidthLimited bandwidth High latencyHigh latency
► Multiple photographs from single locationMultiple photographs from single location Maximizes coverageMaximizes coverage Can be mapped to virtual pan-tilt-zoom camera Can be mapped to virtual pan-tilt-zoom camera
![Page 10: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/10.jpg)
HypothesisHypothesis
►Asynchronicity may improve performanceAsynchronicity may improve performance Helps guarantee coverageHelps guarantee coverage Can review imagery on demandCan review imagery on demand
►Asynchronicity may reduce mental Asynchronicity may reduce mental workloadworkload Only navigation must be done in real-timeOnly navigation must be done in real-time Search becomes self-pacedSearch becomes self-paced
![Page 11: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/11.jpg)
USARSimUSARSim
[http://www.sourceforge.net/projects/usarsim]
► Based on Based on UnrealEngine2UnrealEngine2
► High-fidelity physicsHigh-fidelity physics► ““Realistic” Realistic”
renderingrendering CameraCamera Laser scanner Laser scanner
(LIDAR)(LIDAR)
![Page 12: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/12.jpg)
MrCSMrCSMMulti-ulti-rrobot obot CControl ontrol SSystemystem
![Page 13: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/13.jpg)
MrCSMrCSMMulti-ulti-rrobot obot CControl ontrol SSystemystem
Waypoint Waypoint NavigationNavigation
TeleoperationTeleoperation
Video/ Video/ Image Image ViewerViewer
Status Status WindowWindow
Map Map OverviewOverview
![Page 14: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/14.jpg)
Pilot StudyPilot Study
► Objective:Objective: Find victims Find victims Mark victims on map Mark victims on map
► Control 4 robotsControl 4 robots Waypoint control (primary)Waypoint control (primary) Direct teleoperationDirect teleoperation
► Explore the mapExplore the map Map generated online w/ Occupancy Grid SLAMMap generated online w/ Occupancy Grid SLAM Simulated laser scannersSimulated laser scanners
![Page 15: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/15.jpg)
Experimental ConditionsExperimental Conditions
10 Victims10 Victims
Arena 2
Arena 1
![Page 16: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/16.jpg)
Experimental ConditionsExperimental Conditions
► Streaming ModeStreaming Mode ► Panorama ModePanorama Mode
Panoramas stored for later viewingPanoramas stored for later viewingStreaming live videoStreaming live video
![Page 17: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/17.jpg)
Experimental ConditionsExperimental Conditions(Streaming Mode)(Streaming Mode)
![Page 18: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/18.jpg)
Experimental ConditionsExperimental Conditions(Panorama Mode)(Panorama Mode)
![Page 19: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/19.jpg)
SubjectsSubjects
►21 paid participants21 paid participants 9 male, 12 female9 male, 12 female No prior experience with robot controlNo prior experience with robot control Frequent computer users: 71%Frequent computer users: 71% Played computers games > 1hr/week: Played computers games > 1hr/week:
28%28%
![Page 20: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/20.jpg)
MethodMethod
►Written instructionsWritten instructions►20 min. training session20 min. training session
Both streaming and panoramas enabledBoth streaming and panoramas enabled Encouraged to find and mark at least one Encouraged to find and mark at least one
victimvictim
►20 min. testing session (Arena 1)20 min. testing session (Arena 1)►20 min. testing session (Arena 2)20 min. testing session (Arena 2)
![Page 21: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/21.jpg)
MetricsMetrics
►Switching timesSwitching times►Number of victimsNumber of victims
Thresholded accuracyThresholded accuracy
![Page 22: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/22.jpg)
Victims FoundVictims Found
Average Average # of # of
victims victims foundfound
Accuracy ThresholdAccuracy Threshold
11
22
33
44
55
66
Within Within 0.75m0.75m
Within 1mWithin 1m Within 1.5mWithin 1.5m Within 2mWithin 2m00
PanoramaStreaming
![Page 23: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/23.jpg)
11
22
33
44
55
66
77
First SessionFirst Session Second SessionSecond Session
Panorama First
Streaming First
< 1.5m< 2m
< 1.5m< 2m
00
Trial Order InteractionTrial Order Interaction
AveragAverage # of e # of victimvictim
s s foundfound
![Page 24: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/24.jpg)
Switching TimeSwitching Time (Streaming Mode) (Streaming Mode)
AveragAverage # of e # of reportreport
ed ed victimvictim
ss
1122
2020
Number of SwitchesNumber of Switches
1100
88
66
44
22
4040 6060 8080 100100 12012000
00
p=0.064
![Page 25: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/25.jpg)
Switching TimeSwitching Time (Panorama Mode) (Panorama Mode)
AveragAverage # of e # of reportreport
ed ed victimvictim
ss
1122
2020
Number of SwitchesNumber of Switches
1100
88
66
44
22
4040 6060 8080 100100 12012000
00
![Page 26: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/26.jpg)
SummarySummary
►Streaming is better than panoramicStreaming is better than panoramic Perhaps not by as much as expectedPerhaps not by as much as expected Conditions favorable to streaming videoConditions favorable to streaming video
►Asynchronous performance has Asynchronous performance has potentialpotential May avoid forced pace switchingMay avoid forced pace switching May scale with team sizeMay scale with team size
![Page 27: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/27.jpg)
Synchronous ScalingSynchronous Scaling
► Objective:Objective: Find victims Find victims Mark victims on map Mark victims on map
► Control 4, 8, 12 robotsControl 4, 8, 12 robots Waypoint control (primary)Waypoint control (primary) Direct teleoperationDirect teleoperation
► Explore the mapExplore the map Map generated online w/ Occupancy Grid SLAMMap generated online w/ Occupancy Grid SLAM Simulated laser scannersSimulated laser scanners
![Page 28: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/28.jpg)
Experimental ConditionsExperimental Conditions
8
4
12
![Page 29: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/29.jpg)
Experimental ConditionsExperimental Conditions(Streaming Mode)(Streaming Mode)
![Page 30: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/30.jpg)
SubjectsSubjects
►15 paid participants15 paid participants 8 male, 7 female8 male, 7 female No prior experience with robot controlNo prior experience with robot control Most were frequent computer usersMost were frequent computer users
![Page 31: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/31.jpg)
MethodMethod
►Written instructionsWritten instructions►20 min. training session20 min. training session
Encouraged to find and mark at least one Encouraged to find and mark at least one victimvictim
►20 min. testing session (4 robots)20 min. testing session (4 robots)►20 min. testing session (8 robots)20 min. testing session (8 robots)►20 min. testing session (12 robots)20 min. testing session (12 robots)
![Page 32: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/32.jpg)
MetricsMetrics
►Explored regionsExplored regions►Number of victimsNumber of victims►Neglect toleranceNeglect tolerance►Switching timesSwitching times►Number of missionsNumber of missions►NASA-TLX workloadNASA-TLX workload
![Page 33: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/33.jpg)
Explored RegionExplored Region
Area explore
d
![Page 34: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/34.jpg)
Victims FoundVictims Found
Number of
Victims
![Page 35: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/35.jpg)
Victims Found per RobotVictims Found per Robot
Number of
Victims
![Page 36: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/36.jpg)
Neglected RobotsNeglected Robots
Number of Robots
Totally
Initial Move
![Page 37: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/37.jpg)
Switch TimesSwitch Times
Number of Switches
![Page 38: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/38.jpg)
Mission NumbersMission Numbers
Number of Missions
![Page 39: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/39.jpg)
NASA-TLX WorkloadNASA-TLX Workload
Workload
![Page 40: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/40.jpg)
(Interaction Time)
(Neglect Tolerance)
Fan-outFan-out
![Page 41: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/41.jpg)
SummarySummary
►Bounded number of directly Bounded number of directly controllable robots between 8 and 12controllable robots between 8 and 12 Diminishing returns as robots are addedDiminishing returns as robots are added Performance drops above 8 robotsPerformance drops above 8 robots
►Fan-out parallels the number of robots Fan-out parallels the number of robots operator controlsoperator controls Operators using satisficing strategyOperators using satisficing strategy
![Page 42: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/42.jpg)
Asynchronous Scaling Asynchronous Scaling (Proposed)(Proposed)
► Objective:Objective: Find victims Find victims Mark victims on map Mark victims on map
► Control 4, 8, 12 robotsControl 4, 8, 12 robots Waypoint control (primary)Waypoint control (primary) Direct teleoperationDirect teleoperation
► Explore the mapExplore the map Map generated online w/ Occupancy Grid SLAMMap generated online w/ Occupancy Grid SLAM Simulated laser scannersSimulated laser scanners
![Page 43: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/43.jpg)
Experimental ConditionsExperimental Conditions
8
4
12
![Page 44: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/44.jpg)
Experimental ConditionsExperimental Conditions(Panorama Mode)(Panorama Mode)
![Page 45: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/45.jpg)
MethodMethod
►Written instructionsWritten instructions►20 min. training session20 min. training session
Both streaming and panoramas enabledBoth streaming and panoramas enabled Encouraged to find and mark at least one Encouraged to find and mark at least one
victimvictim
►20 min. testing session (4 robots)20 min. testing session (4 robots)►20 min. testing session (8 robots)20 min. testing session (8 robots)►20 min. testing session (12 robots)20 min. testing session (12 robots)
![Page 46: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/46.jpg)
MetricsMetrics
►Explored regionsExplored regions►Number of victimsNumber of victims►Neglect toleranceNeglect tolerance►Switching timesSwitching times►Number of missionsNumber of missions►NASA-TLX workloadNASA-TLX workload
![Page 47: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/47.jpg)
Expected ContributionsExpected Contributions
►Determine when asynchronicity is Determine when asynchronicity is usefuluseful Advantages for larger team sizesAdvantages for larger team sizes Simultaneous search is not viableSimultaneous search is not viable
►Establish performance baselines for Establish performance baselines for asynchronous searchasynchronous search
![Page 48: Potential Scaling Effects for Asynchronous Video in Multirobot Search](https://reader036.fdocuments.us/reader036/viewer/2022062500/5681593f550346895dc67e05/html5/thumbnails/48.jpg)