RoboSub Simulator - San Diego Mesa College · Engineering, Project Management, and working as a...
1
RoboSub Simulator Abstract: Training Artificial Neural Networks in Virtual Environments The author demonstrates a novel workflow for training robots to recognize objects and optimize autonomous vehicle navigation. This workflow combines several neural network frameworks along with a customized parametric 3D rendering pipeline to harden the resulting networks against dynamic environmental variables. The author used validation data from the actual robotics competition to verify the object recognition network generalized to real world conditions. To facilitate vehicle autonomy, a Deep Q Reinforcement-Network(DQN) was limited to sensory input available under real world conditions, and degraded to simulate actual sensor noise. During training, the reward function was driven by accurate ground truth data available as a consequence of the virtual environment. Network outputs will need to be matched to actual vehicle dynamics to minimize variation between the simulated environment and real world conditions. Problem Solving: Approaching Vehicle Autonomy Artificial Neural Networks(ANNs) are not new to computers, however recent advancements(Mnih , et al. 2015) in their application have demonstrated capabilities beyond human ability for a narrow set of problems. To automate a robot submarine for the purpose of the AUVSI Foundation’s Robosub competition, there are multiple obstacles to robust vehicle navigation which must first be overcome. “Localization” or identifying the vehicle’s location is possibly the most difficult. Many teams elect to navigate by landmark identification using ambient light and digital imaging sensors. Translating identification of landmarks in to thruster outputs is a two part process involving “mission planning” as well as closed loop controllers to address “inverse kinematics”. This poster details the intermediate steps which were required in order to teach a submarine to learn under minimal supervision. Acknowledgements: “On Behalf of the Mesa College MATLAB Club I would like to thank…” The AUVSI Foundation for putting on the Robosub competition Club-Advisor and Math Professor Ken Kuniyuki San Diego Mesa College Inter-Club Council (ICC) San Diego Mesa College Associated Student Government San Diego Mesa College / San Diego City College School District Coleman University Linn-Benton Community College , Albany OR San Diego Mesa College School of Mathematics and Natural Sciences The departments of Physical Sciences, Computer Science, and Student Services for their encouragement and support. Special thanks to my family, Dr. James Burns, Terry Ireland, Bradley Risse, Mark Halls, Alex Gilbert, Adam Thorpe, Phelps Witter IV, and the rest of my friends for their generous support enabling all past and future work. Autonomous Submarines References: "Little Known Facts About The Ocean - MarineBio.org". MarineBio Conservation Society. Web. Accessed 4:32 AM 4/29/2016. <http://marinebio.org/marinebio/facts/>. Figure 1) Ringo. "Bitkili Tankda Led Kullanımı." Bitkili Tankda Led Kullanımı. N.p., 20 Nov. 2015. Web. 29 Apr. 2016. Figure 2) Saders, John Van. "Climbing the Learning Curve." Ordino. Ordino, 9 Jan. 2015. Web. 29 Apr. 2016. http://auxilus.com/category/books/ "Teledyne RD Instruments Co-Sponsors." Press Release Distribution, EDGAR Filing, XBRL, Regulatory Filings. Businesswire, 14 July 2015. Web. 29 Apr. 2016. Figure 8) Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., and Torr, P. Conditional random fields as recurrent neural networks. arXiv:1502.03240, 2015. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (Dec 2013). Playing Atari with deep reinforcement learning. Technical Report arXiv:1312.5602 [cs.LG], Deepmind Technologies. Qiao Wang, et al. "Virtual Worlds as Proxy for Multi-Object Tracking Analysis" CVPR, Las Vegas, Nevada, USA; June 26 - July 1st, 2016. Mnih, Volodymyr, et al. "Asynchronous Methods for Deep Reinforcement Learning." arXiv preprint arXiv:1602.01783 (2016). Figure 1) Although some visible-light penetrates through the surface of the ocean, sea water is virtually opaque to most of the EM spectrum. Helping Find Answers To Questions On March 8, 2014 (17:22 UTC), a Boeing 777 carrying 239 people disappeared from military RADAR. Over two years have passed since MH370 disappeared. Autonomous underwater vehicles (AUVs) have been quietly searching for the aircraft’s wreckage ever since. It’s still unknown what transpired, but hopefully these AUVs will help bring closure to those who lost family. Robotics operates at the intersection of Computer Science, Mechanical, and Electrical Engineering. Building a team that possesses all of these necessary skills can be challenging. These problems inspired the development of a software simulation intended to substantially lower the barriers to entry and logistics expenses involved in testing hypothesis. Accelerating STEM Education Figure 2) A simulated environment enables students to apply the knowledge they’ve learned in the classroom at no cost. By accelerating this “cycle time”: inspiration can from “idea” to “experiment” move rapidly. In a virtual learning environment: the experimental conditions are passively stable. The world’s oceans effectively conceal their contents from observation. High frequency sound waves, and disturbances in the earth’s magnetic field, provide what few clues are available to measure the ocean floor. “Less than 10% of that space has been explored by humans. 85% of the area and 90% of the volume constitute the dark, cold environment we call the deep sea. The average depth of the ocean is 3,795m” Source: MarineBio.org Autonomous Submarines provide the means to measure the contents of our oceans, delivering high resolution sonar imaging sensors to depths that can’t be reached from surface vessels, and covering expanses of ocean too vast to feasibly travel by surface ship. This environment is hostile and unforgiving. Without GPS, high speed communications, or meaningful landmarks: the depths of the ocean demands no less than the most advanced in autonomous control software for these vehicles to survive their mission. The stated goal of the Robosub competition is: “to develop skill in accomplishing realistic missions with autonomous vehicles and to foster relationships between young engineers and the organizations developing and producing autonomous vehicle technologies.”(Teledyne) Each year, updated rules and task descriptions are published by the competition organizers. Although every year is slightly different, recurring themes involve navigation with passive and active sonar, inertial navigation, and the use of image processing techniques to identify and manipulate brightly colored objects and landmarks. In addition to practicing Digital Signal Processing techniques, Robosub provides students with real world experience in Systems Engineering, Project Management, and working as a member of a team. SD Mesa College Research, San Diego, CA Author: Nicholas Cantrell AUVSI Foundation Robosub Competition “Robosub” is an annual event held at the U.S. Navy Space and Naval Warfare Systems Center Pacific's TRANSDEC facility in San Diego, CA. Image Attribution: AUVSI Foundation's RoboSub Competition Figure 4) Pictured below is a screen capture of the simulator from a random position in the pool. Thousands of images just like this one were used with NVIDIA DIGITS to create the pattern of colorful squares seen in the bottom right corner. After training, these squares are what make up the Convolution Neural Network. Figure 5) This graph shows 30 minutes of training while a Reinforcement Learning Network attempted to learn the distance between two points in Cartesian space(provided only with random sensor inputs, and a reward that increased as the network outputs got closer to the correct solution). Figure 6) Although this GoogLeNet model was trained on an insufficiently large dataset to perform at near- human performance, code-efficiency improvements are currently underway to produce and sort training images much faster. Figure 7) Diagnostic Data streaming out of the browser’s JavaScript console. Looking at this data, can you predict the reward function? yaw dept h dest X dest Y destz Current Position(X,Y,Z,ψ) Desired Position(X,Y,Z) Output Actions(PWM outputs) Deep Q-Reinforcement Network Image Data 1 (Front) Image Data 2 (Bottom) 2x Reinforcement Networks (Shared Inputs) Mission Controller/Simulator/Sensor Data A common approach to programming a robot to interact with its environment relies on manipulating image data digitally. Image resizes, followed by color space conversions, filtering, and eventually: binary-thresholding allow programmers to emphasize pixels of interest. Conventional Approaches to Robosub and Visual-Servoing A “depth image” generated by the simulator. An example of color based thresholding in LAB colorspace. Figure 3) Shows a Neural Network topology designed by the author to deconstruct and solve the three problems of localization, mission planning, and inverse kinematics independently. Automatic labeling of ground-truth image data is made possible as a result of generating virtual training images. To maximize the utility of the resulting data, the outputs emulate the format of the PASCAL 2012 Image Segmentation dataset. Conventional Approaches to Robosub and Visual-Servoing Figure 8) Comparison of several open-source Neural Network algorithms to illustrate the benefit of PASCAL 2012 formatted data. Image Credit: (Zheng S, et al. 2015) Figure 9) Simulated render s in this format can generate 30 training images per second on commodity consume hardware. posX Hidden Layer Input Layer Output Layer Figure 10) Profiling Code execution demonstrates room for additional processing at 30 frames per second. Image Attribution: John Van Saders Figure 11) Simulink Model describing the simulated IMU Sensor noise
Transcript of RoboSub Simulator - San Diego Mesa College · Engineering, Project Management, and working as a...
RoboSub Simulator Abstract: Training Artificial Neural Networks in Virtual Environments
The author demonstrates a novel workflow for training robots to recognize objects and optimize autonomous vehicle navigation. This workflow combines several neural network frameworks along with a customized parametric 3D rendering pipeline to harden the resulting networks against dynamic environmental variables. The author used validation data from the actual robotics competition to verify the object recognition network generalized to real world conditions. To facilitate vehicle autonomy, a Deep Q Reinforcement-Network(DQN) was limited to sensory input available under real world conditions, and degraded to simulate actual sensor noise. During training, the reward function was driven by accurate ground truth data available as a consequence of the virtual environment. Network outputs will need to be matched to actual vehicle dynamics to minimize variation between the simulated environment and real world conditions.
Problem Solving: Approaching Vehicle Autonomy Artificial Neural Networks(ANNs) are not new to computers, however recent advancements(Mnih , et al. 2015) in their application have demonstrated capabilities beyond human ability for a narrow set of problems. To automate a robot submarine for the purpose of the AUVSI Foundation’s Robosub competition, there are multiple obstacles to robust vehicle navigation which must first be overcome. “Localization” or identifying the vehicle’s location is possibly the most difficult. Many teams elect to navigate by landmark identification using ambient light and digital imaging sensors. Translating identification of landmarks in to thruster outputs is a two part process involving “mission planning” as well as closed loop controllers to address “inverse kinematics”. This poster details the intermediate steps which were required in order to teach a submarine to learn under minimal supervision.
Acknowledgements: “On Behalf of the Mesa College MATLAB Club I would like to thank…”
The AUVSI Foundation for putting on the Robosub competition Club-Advisor and Math Professor Ken Kuniyuki San Diego Mesa College Inter-Club Council (ICC) San Diego Mesa College Associated Student Government San Diego Mesa College / San Diego City College School District Coleman University Linn-Benton Community College , Albany OR San Diego Mesa College School of Mathematics and Natural Sciences The departments of Physical Sciences, Computer Science, and Student Services for their encouragement and support. Special thanks to my family, Dr. James Burns, Terry Ireland, Bradley Risse, Mark Halls, Alex Gilbert, Adam Thorpe, Phelps Witter IV, and the rest of my friends for their generous support enabling all past and future work.
Autonomous Submarines
References: "Little Known Facts About The Ocean - MarineBio.org". MarineBio Conservation Society. Web.
Accessed 4:32 AM 4/29/2016. <http://marinebio.org/marinebio/facts/>. Figure 1) Ringo. "Bitkili Tankda Led Kullanımı." Bitkili Tankda Led Kullanımı. N.p., 20 Nov. 2015.
Web. 29 Apr. 2016. Figure 2) Saders, John Van. "Climbing the Learning Curve." Ordino. Ordino, 9 Jan. 2015. Web. 29
Apr. 2016. http://auxilus.com/category/books/ "Teledyne RD Instruments Co-Sponsors." Press Release Distribution, EDGAR Filing, XBRL,
Regulatory Filings. Businesswire, 14 July 2015. Web. 29 Apr. 2016. Figure 8) Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C.,
and Torr, P. Conditional random fields as recurrent neural networks. arXiv:1502.03240, 2015. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M.
(Dec 2013). Playing Atari with deep reinforcement learning. Technical Report arXiv:1312.5602 [cs.LG], Deepmind Technologies.
Qiao Wang, et al. "Virtual Worlds as Proxy for Multi-Object Tracking Analysis" CVPR, Las Vegas, Nevada, USA; June 26 - July 1st, 2016.
Mnih, Volodymyr, et al. "Asynchronous Methods for Deep Reinforcement Learning." arXiv preprint arXiv:1602.01783 (2016).
Figure 1) Although some visible-light penetrates through the surface of the ocean, sea water is virtually opaque to most of the EM spectrum.
Helping Find Answers To Questions
On March 8, 2014 (17:22 UTC), a Boeing 777 carrying 239 people disappeared from military RADAR. Over two years have passed since MH370 disappeared. Autonomous underwater vehicles (AUVs) have been quietly searching for the aircraft’s wreckage ever since. It’s still unknown what transpired, but hopefully these AUVs will help bring closure to those who lost family.
Robotics operates at the intersection of Computer
Science, Mechanical, and Electrical Engineering. Building a team that possesses all of these
necessary skills can be challenging. These problems
inspired the development of a software simulation intended to
substantially lower the barriers to entry and logistics expenses
involved in testing hypothesis.
Accelerating STEM Education
Figure 2) A simulated environment enables students to apply the knowledge they’ve learned in the classroom at no cost. By accelerating this “cycle time”: inspiration can from “idea” to “experiment” move rapidly. In a virtual learning environment: the experimental conditions are passively stable.
The world’s oceans effectively conceal their contents from observation. High frequency sound waves, and disturbances in the earth’s magnetic field, provide what few clues are available to measure the ocean floor.
“Less than 10% of that space has been explored by humans. 85% of the area and 90% of the volume constitute the dark, cold environment we call the deep sea. The average depth of the ocean is 3,795m” Source: MarineBio.org
Autonomous Submarines provide the means to measure the contents of our oceans, delivering high resolution sonar imaging sensors to depths that can’t be reached from surface vessels, and covering expanses of ocean too vast to feasibly travel by surface ship. This environment is hostile and unforgiving. Without GPS, high speed communications, or meaningful landmarks: the depths of the ocean demands no less than the most advanced in autonomous control software for these vehicles to survive their mission.
The stated goal of the Robosub competition is: “to develop skill in accomplishing realistic missions with autonomous vehicles and to foster relationships between young engineers and the organizations developing and producing autonomous vehicle technologies.”(Teledyne) Each year, updated rules and task descriptions are published by the competition organizers. Although every year is slightly different, recurring themes involve navigation with passive and active sonar, inertial navigation, and the use of image processing techniques to identify and manipulate brightly colored objects and landmarks. In addition to practicing Digital Signal Processing techniques, Robosub provides students with real world experience in Systems Engineering, Project Management, and working as a member of a team.
SD Mesa College Research, San Diego, CA Author: Nicholas Cantrell
AUVSI Foundation Robosub Competition
“Robosub” is an annual event held at the U.S. Navy Space and Naval Warfare Systems Center Pacific's TRANSDEC facility in San Diego, CA.
Image Attribution: AUVSI Foundation's RoboSub Competition
Figure 4) Pictured below is a screen capture of the simulator from a random position in the pool. Thousands of images just like this one were used with NVIDIA DIGITS to create the pattern of colorful squares seen in the bottom right corner. After training, these squares are what make up the Convolution Neural Network.
Figure 5) This graph shows 30 minutes of training while a Reinforcement Learning Network attempted to learn the distance between two points in Cartesian space(provided only with random sensor inputs, and a reward that increased as the network outputs got closer to the correct solution).
Figure 6) Although this GoogLeNet model was trained on an insufficiently large dataset to perform at near-human performance, code-efficiency improvements are currently underway to produce and sort training images much faster.
Figure 7) Diagnostic Data streaming out of the browser’s JavaScript console. Looking at this data, can you predict the reward function?
yaw
depth
destX
destY
destz
Current Position(X,Y,Z,ψ)
Desired Position(X,Y,Z)
Output Actions(PWM outputs)
Deep Q-Reinforcement Network
Image Data 1 (Front)
Image Data 2 (Bottom)
2x Reinforcement Networks (Shared Inputs)
Mission C
ontroller/Sim
ulator/Sensor D
ata
A common approach to programming a robot to interact with its environment relies on manipulating image data digitally. Image resizes, followed by color space conversions, filtering, and eventually: binary-thresholding allow programmers to emphasize pixels of interest.
Conventional Approaches to Robosub and Visual-Servoing
A “depth image” generated by the simulator.
An example of color based thresholding in LAB colorspace.
Figure 3) Shows a Neural Network topology designed by the author to deconstruct and solve the three problems of localization, mission planning, and inverse kinematics independently.
Automatic labeling of ground-truth image data is made possible as a result of generating virtual training images. To maximize the utility of the resulting data, the outputs emulate the format of the PASCAL 2012 Image Segmentation dataset.
Conventional Approaches to Robosub and Visual-Servoing
Figure 8) Comparison of several open-source Neural Network algorithms to illustrate the benefit of PASCAL 2012 formatted data. Image Credit: (Zheng S, et al. 2015)
Figure 9) Simulated render s in this format can generate 30 training images per second on commodity consume hardware.
posX
Hidden Layer Input Layer
Output Layer
Figure 10) Profiling Code execution demonstrates room for additional processing at 30 frames per second.
Image Attribution: John Van Saders
Figure 11) Simulink Model describing the simulated IMU
Sensor noise
