EssEntial REsEaRch: Significant Revenue Opportunities for

EssEntial REsEaRch : Significant Revenue Opportunities for Vision in Service Robots

this 120-page report:

• forecasts the volume of vision system sales expected in 17 market segments and 53 sub-markets, based on proprietary research and interviews

• includes a comprehensive review of technologies, markets and opportunities for vision in service robots not readily available through online search or other sources

• provides an in-depth review of aerospace, land and water applications

This report is an essential tool for suppliers of vision components and systems, vision subsystem integrators, technology researchers, universities looking to commercialize their intellectual property, service robot manufacturers, subsystem OEMs, turnkey solution integrators, the financial investment community, and companies and organizations interested in entering the service robot market.

learn more: http://www.vision-systems.com/research-reports.html

The VISION FOR SERVICE ROBOTS report from Vision Systems Design identifies

multiple near-term market opportunities for vision components and systems in service

robots, totaling several billion dollars, and even greater opportunities in the longer term.

http://www.vision-systems.com/research-reports.html


Vision for Service Robots is a NEWREPORT from Vision Systems Design thatis based on extensive interviewsand original research compiled overeight months by Adil Shafi, well-knownauthority on service robots and presidentof Advenovation, and Conard Holton,Editor of Vision Systems Design. Thisground-breaking report includes detailedand specific application descriptionsfor 17 markets and 53 sub-markets divided

into air-, land- and water-based categories.The current value of installed visionsystems, estimated product and component pricing for each application, and the forecasted number and value of vision systems for each category for 2010 to 2013 and beyond are provided. This unique report provides a comprehensive view of the market for vision in service robots that is not available through online search or other sources.

Executive summary

to order the Vision for service Robots report • Go to http://qmags.com/VSR to order the

full report or http://qmags.com/VSRE to order the Global Forecast excerpt

• Enter your credit card information and purchase the report online

• A single-use PDF will be delivered to the ordering computer by email

• Cost for the full report is $1,995; cost for the global forecast section only is $995

• For help ordering the report, contact Debbie Bouley at [email protected].

Adil Shafi and

Conard Holton

Published by

Current and

Emerging

Technologies,

Applications,

and Markets

Vision for Service Robots

research

Many countries and regions are avidly fostering research and development in this field. They

are doing so because, while industrial robots are sold in the tens of thousands per year, service

robots will offer a volume of unit sales ultimately in the millions. Many of the companies,

universities, labs, and government organizations developing service robots believe that vision

will provide a critical technical and market advantage. This is an exciting and dynamic time for

service robots. With the help of machine vision, these robots are gaining widespread commercial

and personal acceptance as they assist humans in almost every aspect of life.

2

http://qmags.com/VSR

http://qmags.com/VSRE

mailto:debbieb%40pennwell.com?subject=

106 Vision for Service Robots Current and Emerging Technologies, Applications, and Markets Vision systems Design

(ARTAS System) that enables follicular unit extraction for hair transplantation. ARTAS automates, under the direction of a physician, the follicular unit extraction (FUE) approach to harvesting follicular units. The ARTAS System is an investigational device. It does not have FDA clearance and is not available for sale. Restoration Robotics is actively seeking regulatory clearance.

RoadNarrows RoboticsPhilippe Kervizic, International Sales and Business DevelopmentRobotics and Intelligent Systemshttp://www.roadnarrows.com

RoadNarrows Robotics designs and manufactures robot parts and distributes robotic technology, focusing on technologies for mobile and high-end robotics: vision, sensors, manipulator, robots, and simulation software. The RoboSight Neural Camera, based on a neural technology chip, has capabilities for object recognition in research and industry.

Schunkhttp://www.schunk.com

Schunk Expert Days brings together service robots experts from around the world. The 4th annual meeting was held in February 2011, at Schunk headquarters in Hausen, Germany. The modular robotics company

makes compact and combinable rotary actuators, lightweight manipulators, and servo-electrically actuated grippers.

Sonyhttp://pro.sony.com

Sony cameras have been used by industrial robot suppliers for many years and are now being utilized in service robots.

Toyota Partner Robotshttp://www.toyota.co.jp

Willow Garagehttp://www.willowgarage.com

Willow Garage is a privately funded startup focused on innovating and creating a community of partners. It has developed two robots: the PR2, which is a mobile humanoid robot with integrated vision capability, and the Texai, which is a mobile telepresence robot. The company has also developed an open-source ROS software platform (Robot Operating System); it invites third-party developers to work with ROS and develop custom vertical solutions. Willow Garage has sold a few of the PR2 robots and also donated about 11 units to various robotics-focused institutions.

7.4 Government / NGOs / ResearchAutomated Imaging Association (AIA)Dana Whalls, Vice Presidenthttp://www.machinevisiononline.org

Automated Imaging Association (AIA) was organized to advance the global understanding and implementation of vision and imaging technologies to help members grow their businesses.

Autonomous Underwater Vehicle Applications Center (AUVAC)http://www.auvac.org

An incubator formed by an association of international academic, private sector, and government organizations to advance AUV systems and subsystems.


test, and evaluation stages. Vision has an important role to play both on the surface and underwater, in both cases combining cameras and image processing with multiple other sensors and techniques such as radar, sonar, inertial navigation, GPS navigation (surface only), and simultaneous localization and mapping (SLAM).

Defense-related remotely operated vehicles (ROVs) and autonomous underwater vehicles (AUVs) make up approximately 30% of the total unmanned underwater vehicle (UUV) market, according to naval market analysis firm AMI International (Bremerton, WA, USA). The strongest growth will be in AUVs for mine countermeasures, followed by port and harbor security. The next important development will be to “weaponize” the AUVs. Some of the numbers below assume a 5%/yr growth; AMI expects the military UUV market to grow at 3–10%/yr to 2015.

Commercial—Offshore energy exploration is the primary driver of developments in commercial UUVs, with a global 70/30% split between commercial/military vehicle sales. Spending waxes and wanes with energy prices. According to AMI International, trends in both the military and commercial UUVs will be for deeper diving, modular payloads, and multimission capabilities.

ROVs and AUVs are typically categorized as large (for commercial, military, oil and gas), medium (for research, military, oil and gas), and small (for research, military). A medium-size work-class ROV may have a vision system fielding three color cameras of various types, two monochrome cameras, and a range of underwater lighting including HID and halogen lights. An AUV may have at least two cameras.

Scientific—Applications for UUVs in underwater scientific research are numerous, including research on marine life, seabed formations, and underwater currents and rivers, and for building subsea research sites. Researchers frequently adapt or use commercially available small or medium UUVs. On other hand, UUVs developed for research at places

Water: R30.bb Commercial

Vision Units Sold to end of 2009

Vision Units Forecast 2010–2013

Cost per Vision System

Vision Market Size 2010–2013

Opportunity Timeframe

ROVs: offshore oil and gas drilling, construction, inspection, mainte-nance, repair; salvage and retrieval (for work after identification); cleaning and containment of spills

5775* 1244*

<$92,500(for large, medium,

small RoVs)

<$115m

immediate through leading UUV suppliers, e.g., schilling Robotics, Kongsberg, Hafmynd, sub-Atlantic

AUVs: offshore oil and gas inspection, maintenance; salvage and retrieval (for identification); inspection of aqueducts, sewers

315* 70* $49,500 $3.47m

immediate through leading UUV suppliers, e.g., schilling Robotics, Kongsberg, Hafmynd, Lockheed martin

*Derived from Ami international estimates

to order: http://qmags.com/VSR

The Vision for Service Robots report goes well beyond other reports that have attempted to estimate the size of market opportunities for vision. Our goal is to help readers assess technology developments and market sizes and trends—and then make key decisions regarding revenue opportunities for their companies to pursue.

To help understand the relative sizes of and trends in the widely differing service robot markets, we include tables for each of 17 distinct market segments, with detail for 53 applications or sub-markets. Each table provides more complete market intelligence by including estimates of:

• units sold or initiatives underway in 2009,

• number of units forecasted to be sold (or initiatives underway) from 2010 to 2013,

• cost per vision system (US dollars),

• estimated size of the vision market for each segment (US dollars), and

• the opportunity timeframe for selling vision products for specific applications, ranging from immediate to emerging long term.

Global Forecast for Vision in service Robots

3

also included: Directory of 100+ Relevant OrganizationsThe Vision for Service Robots report includes an extensive directory of organizations involved in service robot manufacturing, research and development as well as an overview of their projects, and links to key resources and websites.



Stereo camerasystem on pan-tilt head

Chin joystick

TFT display

Mini joystick

Tray withIR sensor surface

Panningarm ofTFT display

Panning armof manipulator

1101VSDrobotSec5.7


FRIEND from the IAT, University of Bremen.

meals, medical records, and radiology films. The Pyxis HelpMate SP has a 200-lb payload and various lockable compartments for storage. The color touch-screen allows human operators to send it to virtually any location in a hospital. More than 100 Pyxis HelpMate units have been sold to hospitals within the US.

The original HelpMate used a structured-light vision system, comprising a camera that focused on objects up to 8 ft in front of the robot. The camera differenti-ated between obstacles at various distances and altered the course of the robot to maneuver around them. Two infrared strobe lights, positioned 6 in. and 18 in. off of the

floor, provided planes of lights for the camera to detect objects in front. A laser scanner replaced the structured-light system, providing a wider field of view (180º as opposed to 60º with the camera), finer resolution, and improved reliability.

In other developments, makers of robotics hardware and platforms are beginning to work with software and services companies to overcome some of the obstacles delaying the commercial sale of assistive robots for home use. For example, writing in his web site blog, Hoaloha Robotics (Seattle, WA, USA) founder Tandy Trower describes the collaboration of his software and services company with Robosoft (Bodart, France), which makes a variety of robots.

5.7.2 Wheelchair Robots

Some companies have teamed with univer-sities to develop specific assistive robotic systems that include robotic wheelchairs, such as the Automated Transfer and Retrieval System developed by Vision, Assistive Devices, and Experimental Robotics Laboratory at Lehigh University (Lehigh, PA, USA). It was developed with and is now sold by Freedom Sciences (Philadelphia, PA, USA).

A wheelchair robot developed at the University of Bremen, FRIEND (Functional Robot arm with user-frIENdly interface for Disabled people), is semi-autonomous and designed to support

Kompaï assistive robot from

Robosoft will use software

from Hoaloha Robotics.


road boundaries, such as painted lines and curbs, are being enhanced by lidar to allow for improved positioning on roads running across rough desert terrain.

The iRobot 210 Negotiator is a low-cost, easily portable, tele-operated robot that performs basic reconnaissance, increasing SA in dangerous scenarios for public safety professionals such as police, SWAT teams, first responders, or bomb squads. It has a color video camera and low-light infrared illuminated camera along with an infrared LED array for illu-minating in all ambient light conditions.

5.1.6 Developments in Vision for Unmanned Ground Vehicles

UGVs and SLAM—Vision Robotics Federal Systems (VRFS; San Diego, CA, USA) was contracted by Space and Naval Warfare Command (SPAWAR) Systems Center, Pacific (SSC) to adapt the company’s 3-D vision-based simultaneous localization and mapping (3D VSLAM) software for use on military UGVs.

3D VSLAM is of interest to SSC as an adjunct to other localization and mapping systems currently in use, such as lidar or inertial-based systems. The capability provided by 3D VSLAM is attractive due to its potentially low cost, large field of view, and ability to perform on nonplanar ground surfaces.

The full project and test reports from 2009 may be found by contacting the Joint Ground Robotics Enterprise. In summary, VFRS demonstrated the following capabilities with four stereo sensor pairs mounted on a Segway vehicle:

• accurate 3-D maps of complex cluttered indoor spaces, including office buildings and bunkers• 3-D visualization capability sufficient for an operator to identify large objects such as

chairs, tables, etc.• mapping speeds in excess of 500 m2 per hour• autonomous navigation speeds in excess of 1.5 mph, and tele-op mapping speeds of 3 mph• robustness to varying lighting, poor vehicle traction, or difficult obstacles• the ability to map 10+ rooms in a single, large building, and operate continuously for

more than one hour

While the VRFS system is among the more mature and robust visual SLAM systems under development, it is not yet mature enough to be deployed. Shortcomings of the current system include:

• excessive weight and power requirements for a man-portable vehicle• requirement for four stereo pairs requires considerable mechanical and electrical

modification to existing vehicles• considerable computational requirements exceed realistic compatibility with

existing vehicles• unproven capability operating over rough terrain• unproven capability in outdoor environments• unproven capability in operating over large areas (multiple buildings, tunnels, etc.)

3D VSLAM is of interest

as an adjunct to other

localization and mapping

systems currently in

use, such as lidar and

inertial-based systems.

It is potentially low cost,

has a large field of view,

and can perform on

nonplanar surfaces.

Negotiator from iRobot.

land applications• Defense and Security: surveillance, EOD, combat,

transport (military UGVs); surveillance (civilian); explosive or hazard disposal (civilian); explosive or hazard disposal (de-mining); firefighting, law enforcement

• Farming: ground-based farming, livestock manage-ment, harvesting; tree-based produce retrieval; feeding and cleaning in zoos; wildlife conservation

• Food Processing: dairy (milking and cleaning);meat (processing); poultry (chicken and egg handling); fruit and vegetable handling and sorting

• Transportation: assistive (cars, trucks; autonomous or following (cars, trucks)

• Mining, Construction, Maintenance: under- and above-ground mining; construction and demolition (including nuclear); roads and highways construction and maintenance; railway operations; facilities and plants inspection and maintenance; runways and gates at airports; ports, docks, locks and canals (including logistics); nuclear operations, inspection

• Office and Warehouse: mobile (communications,telepresence, delivery, courier (includes hospitals); warehousing logistics

• Health Care: assistive (reach and access, switchingpositions, wheelchairs, walking); diagnostics (scanning robots or automated scan-based robotic motion); telepresence (roaming doctor, tracking, doctor-patient interaction); logistics

• Health Rehabilitation: locomotion (exertive strength or walking balance); retraining (hand-eye retraining with 1) force or 2) vision)

• Health Surgical

• Education and Entertainment: educational andresearch; toys; kiosk/marketing; coasters/rides

• Home Convenience: cleaning (roof, gutters, driveway, pool, lawn, snow, windows, walls, floors); mobile robot helper, companion (wheeled, humanoid); home surveillance and security

Market Opportunities The Vision for Service Robots report divides the market opportunities for vision into three sections that include details on 17 markets and 53 submarkets:

4


5


Onboard processing saves critical time for reprogramming; for example, the recent Deep Impact spacecraft mission past comet Hartley 2 had a 10-hour observation window and it was helpful to be able to make onboard decisions. The imaging system was developed by Ball Aerospace & Technologies (Boulder, CO, USA) and contained the High Resolution Instrument (HRI), which comprised a telescope with a 30-cm aperture, an infrared spec-trometer, and a multispectral 1024 × 1024-pixel CCD camera with filter wheel. The HRI CCD camera images a comet with less than 2-m-per-pixel scale when the flyby spacecraft is 700 km away. The Medium Resolution Instrument (MRI) serves as the functional backup for the HRI and takes images with a scale of 10 m per pixel in the visible spectrum at 700 km.

4.1.2 Autonomous VehiclesResearchers at numerous universities and scientific centers around the world are using machine-vision cameras and systems to develop prototypes and prepare for future robotic missions on other planets. For example, researchers at the Autonomous Space Robotics Lab at the University of Toronto (Toronto, ON, Canada) developed a visual path-following technique to enable long-range autonomous navigation of a six-wheeled rover platform.

The researchers used a Bumblebee XB3 stereo camera from Point Grey Research (Richmond, BC, Canada) as the only sensor for local-ization and mapping. Using this technique, a rover may autonomously traverse a multi-kilometer route in unstructured, 3-D terrain, without an accurate global reconstruction.

The pace of space exploration is increasing as space launches and exploration are becoming commercial ventures, as shown by the December 2010 launch and recovery of the SpaceX Dragon rocket from Cape Canaveral. Recently, Google announced a commercial incentive to perform space exploration via their Lunar X Prize compe-tition, a $30 million international competition to safely land a robot on the surface of the Moon, travel 500 m over the lunar surface, and send images and data back to Earth.

About two dozen organizations have signed up for this program. One notable example, Astrobotic Technology (Pittsburgh, PA, USA), is a Carnegie Mellon University (CMU) spin-off company that has Caterpillar as a sponsor for its first robotic expedition to the moon. The initial Astrobotic mission is scheduled to revisit the Apollo 11 site in April 2013 with a 5-ft-tall, 160-lb robot broadcasting high-definition 3-D video from its camera mast. The mission will carry payloads to the Moon and convey the experience to the world via Internet video access.

Vision Market PerspectivesNASA’s space program evolves continually each year, as it has for almost four decades, subject to the vagaries of federal budgetary pressures. There is a growing emphasis to focus on unmanned missions with robots to far away heavenly bodies as opposed to manned missions to nearby heavenly bodies (at present nothing beyond the Moon or Mars is considered).

Researchers at the University of Toronto are using only

a single stereo camera for localization and mapping for

a planet-exploring robot.

Third Red Rover prototype from Astrobotic Technology, with

camera mast in center.


efficiency aspheric optical system that provides near-SIT (silicon intensified target) performance from a nonintensified CCD sensor

• OE15-358 B&W camera for general-purpose inspection• a range of underwater lighting including HID and halogen lights

The Marlin class of AUVs from Lockheed Martin (Bethesda, MD, USA) is used in offshore oil and gas, science, oceanography, military, and other applica-tions. The AUV ranges in length from 3 ft to more than 16 ft depending on the capability required. Vertical thrusters enable operation in lakes, rivers, estuaries, harbors, ports, coastal regions, and oceans.

The Marlin’s high-resolution optical and acoustic sensor package can provide 3-D pictures up to 1000 ft beneath the surface and rapidly inspect for potential damage. Its top speed is 4 knots and it can cruise for up to 16 hr.

Lockheed Martin argues that while AUVs have demonstrated solid performance with simple autonomy for missions such as bathymetric survey

and high-resolution sonar imaging, AUVs in the future must demonstrate much higher levels of autonomy to perform IMR tasks. In the near term, AUVs could be deployed from small utility vessels (eliminating the need for large support vessels required by ROVs), be capable of operations in higher seas, and perform simple IMR tasks much more effi-ciently without the operational limitations and equipment hazards imposed by umbilical and tether management systems. Eventually, AUVs would reside in the subsea field for months, further reducing costs while improving performance and safety.

AUVs have uses inland as well. For example, the New York City Department of Environmental Protection chose the Woods Hole Oceanographic Institution (WHOI; Woods Hole, MA, USA) to use REMUS (a 9.5-ft-long AUV known in this application as the Tunnel Inspection Vehicle), to inspect a leaking tunnel aqueduct that brings water from a reservoir in upstate New York to the city. Since the developers required images for the full 360° in the tunnel, they installed five DVC-1412M cameras from DVC (Austin, TX, USA), each equipped with a 4.8-mm lens, to provide an 85° field of view. Four strobe lights illuminated the surfaces and were synchronized with the cameras and frame grabber using Matrox Imaging (Dorval, QC, Canada) MIL-Lite imaging library. The Tunnel Inspection Vehicle developed by WHOI for NYC.

Marlin from Lockheed Martin can autonomously locate

cable for recovery.


aerospace applications• Spacecraft: robotic spacecraft; autonomous

vehicles (rovers)

• Satellites

• Aircraft: unmanned drones for surveillance,combat, research; refueling robots aboard aircraft (airborne, including UAV to UAV refueling); resupplying robots aboard aircraft carriers; MAVs and flying swarms for reconnaissance Land Applications

Water applications• Defense and Security: surface (mine counter-

measures, intelligence, surveillance, reconnais-sance, anti-submarine missions, harbor inspection, law enforcement, and naval logistics and support); underwater (mine countermeasures, intelligence-surveillance, reconnaissance, antisubmarinemissions, harbor inspection, search and rescue, and naval logistics and support)

• Commercial: ROVs (offshore oil and gas drilling,construction, inspection, maintenance, repair, salvage and retrieval (for work after identification), cleaning and containment of spills); AUVs (offshore oil and gas inspection, maintenance, salvage and retrieval (for identification), inspection of aqueducts, sewers)

• Scientific: ROVs (research marine environment);AUVs (research marine environment)

Market Opportunities Continued:


103 Vision for Service Robots Current and Emerging Technologies, A

pplications, and Markets

Vision systems Design

Rutgers University

RUCOOL Coastal Ocean Observation Lab—

Institute of Marine and Coastal Sciences

http://rucool.marine.rutgers.edu

SLOCUM underwater robot—video

Stanford Research Institute

Centibots

Karto SDK

Stanford University

http://www.stanford.edu

SAIL—Stanford Artificial

Intelligence Laboratory

Professor Sebastian Thrun

http://robotics.stanford.edu

DARPA Urban Challenge

Stanford University’s work in robotics is

often at the leading edge of innovation. Most

recently, Stanford has distinguished itself

in mobile robotics through a winning entry

in the DARPA-sponsored Grand Challenge.

The contest involved the design, production,

and racing of an autonomous vehicle that

could traverse the Mojave Desert under

varying land conditions. They key to Stanford

University’s winning entry was a fast sensor-

fusion-based solution with a very fast top-

level situational analysis capability, which

enabled the appropriate vision approach for

navigation and steering.

University of California–Berkeley

http://robotics.eecs.berkeley.edu

University of Washington

Seaglider—video

USC Robotics Research Lab

http://robotics.usc.edu

Waseda University

Department of Modern

Mechanical Engineering

School of Creative Science and Engineering

Dr. Shigeki Sugano, Professor

http://www.sugano.mech.waseda.ac.jp

7.2 Competitions

AAAI Robotic Awards

http://www.aaai.org

First International

Humanoid Robot Olympics

http://news.xinhuanet.com

FIRST Robotics Competition

http://www.usfirst.org

IGVC

Intelligent Ground Vehicles Competition

http://www.igvc.org

Competitions for ground military efforts,

held at Oakland University each year; done

amongst about 50 universities each year.

RoboCup

http://www.robocup.org

Smart Cameras Make Soccer Robots

Intelligent: RoboCup Winners

Stanford University















7.2 Competitions

AAAI Robotic Awards

First International


http://news.xinhuanet.com RoboCup

http://www.robocup.org

Smart Cameras Make Soccer Robots

http://www.aaai.org

First International




RoboCup

7.2 Competitions

AAAI Robotic Awards

http://www.aaai.org Competitions for ground military efforts,


amongst about 50 universities each year.7.2 Competitions

AAAI Robotic Awards

http://www.aaai.org


http://www.igvc.org

Competitions for ground military efforts,



Stanford University















7.2 Competitions

Stanford University















7.2 Competitions

7.2 Competitions

AAAI Robotic Awards

IGVC


http://www.igvc.org

























7.2 CompetitionsIGVC













in the DARPA-sponsored Grand Challenge. often at the leading edge of innovation. Most





School of Creative Science and Engineering






http://robotics.usc.edu

Waseda University

Department of Modern Waseda University

Department of Modern


Rutgers University







Centibots



















97

Vision for Service Robots Current and Emerging Technologies, Applications, and Markets

VISION SYSTEMS DESIGN

6.4 Commercial

Commercial applications of UUVs, particularly ROVs, are driven mainly by tasks related

to drilling and construction, and inspection, maintenance, and repair (IMR) for the

o� shore oil and gas industry, submarine cable instal-

lation and maintenance, harbor maintenance, and

accident response.

� e subsea damage containment and repair

undertaken following the Deepwater Horizon Gulf

oil spill in April 2010 was performed by several dozen

ROVs, along with several scienti� c AUVs research-

ing the impact on the environment. � e ROVs were

usually equipped with dual manipulators and were

of a group known as work-class ROVs (WROVs).

One of the � rst on the scene was the Ultraheavy Duty

(UHD) WROV from Schilling Robotics (Davis, CA,

USA), which is capable of arduous tasks that require

signi� cant power to li� , position, and install subsea � eld equipment. � e

UHD comes with a digital telemetry system that features a GigE backbone

and accommodates a variety of vision systems

At the other end of the scale is the LBV series of miniROVs from

SeaBotix (San Diego, CA, USA) (now owned by Bolt Technology). � e

SeaBotix LBV200-4, for example, is capable of a diverse range of applica-

tions, including aquaculture,

in-shore and o� -shore tasks,

search and rescue, and

security. In addition to its side-scan sonar, it

has a camera system with 180° tilt, 270° range

of view, 560-line-wide dynamic range color

camera, and uses NTSC or PAL video format.

� e lighting is a 700-lumen LED array.

� e vision systems for ROVs can be

extensive—for example, Eastar O� shore

(Singapore) is equipping its latest ROVs for the oil

and gas industry with this suite of cameras developed by

Kongsberg Maritime (Kongsberg, Norway):

• OE14-366 color zoom camera, which uses the latest super HAD CCD technology to

provide excellent light sensitivity and image de� nition

• OE14-372 � xed-focus color camera, which uses interline transfer CCD solid-state sensors

to produce excellent light sensitivity and image de� nition

• OE14-110 compact color camera, which is a cost-e� ective, miniature color camera with

excellent light sensitivity and image de� nition

• OE15-100 B&W navigation camera with advanced signal processing coupled with a high-

UHD WROV and

pilot’s control panel (top)

from Schilling Robotics.

LBV 150 from SeaBotix.

VISION SYSTEMS DESIGN




Current and Emerging Technologies, Applications, and Markets















(Kongsberg, Norway):








LBV 150 from SeaBotix.






































Kongsberg Maritime

Kongsberg Maritimeprovide excellent light sensitivity and image de� nition





Kongsberg Maritime

Kongsberg Maritime










Kongsberg Maritime

Kongsberg Maritime (Kongsberg, Norway):


















The most utopian of these goals is to take the human being out of the picture altogether and to let these vehicles drive in a controlled, error-free manner while maximizing efficiency and minimizing pollution. Beyond the legal and psychological complications, the obvious traffic challenges are made more difficult by uneven terrain, weather, visibility, light, and the cost of making such profound changes. Nevertheless, numerous efforts are underway to make change possible.

5.4.1 Developments in Robotic TransportationResearch on autonomous cars has been underway since the mid-1960s, but the most important innovations occurred in 2004, when the US Defense Advanced Research Projects Agency (DARPA) issued the first of its Grand Challenges. Stanford won in 2005 with a drive of 132 miles through the California desert. In 2007, a similar urban challenge proved that autonomous vehicles could navigate city streets.According to the World Health Organization, more than 1.2 million lives are lost every year

in road traffic accidents. As part of its research to reduce this number and lower the environ-mental impact of traffic, Google has a small fleet of autonomous cars that has driven 140,000 miles on California highways, with a driver ready to take over if needed. The company believes this technology has the potential to reduce fatalities, perhaps by as much as half.

The company is also confident that self-driving cars will transform carpooling, signifi-cantly reducing vehicle usage, as well as help create the new “highway trains of tomorrow.” These highway trains should cut energy consumption while also increasing the number of people that can be transported on major roads.

Google uses a modified Toyota Prius with an array of sensors, including a rotating lidar sensor on the roof, a video camera near the rearview mirror, a position sensor, and radar, along with a GPS receiver and inertial motion sensor. The leader of the project, Sebastian Thrun, director of the Stanford Artificial Intelligence Laboratory (Stanford, CA, USA), says even the most optimistic predictions put the deployment of the technology more than 8 years away. Legal safety issues will be a concern since current laws say a human must be in control of a car at all times.Sensor fusion with vision enablers is essential to make transportation

assistive or autonomous. As was illustrated most dramatically by the Stanford team when they won the DARPA Challenge, the key to their success was sensor fusion. They utilized lidar, 3-D triangulation, and other sensors, and had a master controller that kept track of the vehicle’s mode in real time (e.g., on a fast flat road or uneven terrain). This master assessment then trigged the appropriate sensors (including vision) to perform the proper actions.

Google autonomous car driving in the Bay area.

assistive or autonomous. As was illustrated most dramatically by the Stanford team when they won the DARPA Challenge, the key to their success was sensor fusion. They utilized lidar, 3-D triangulation, and other sensors, and had a master controller that kept track of the vehicle’s mode in real time (e.g., on a fast flat road or uneven terrain). This master assessment then

assistive or autonomous. As was illustrated most dramatically by the Stanford team when they won the DARPA Challenge, the key to their success was sensor fusion. They utilized lidar, 3-D triangulation, and other sensors, and had a master controller that kept track of the vehicle’s

is essential to make transportation assistive or autonomous. As was illustrated most dramatically by the Stanford team when they won the DARPA Challenge, the key to their success was sensor fusion. They utilized lidar, 3-D triangulation, and other sensors, and had a master controller that kept track of the vehicle’s

Sensor fusion with vision enablers is essential to make transportation assistive or autonomous. As was illustrated most dramatically by the Stanford team when they won the DARPA Challenge, the key to their success was sensor fusion. They utilized lidar, 3-D

Sensor fusion with vision enablers is essential to make transportation assistive or autonomous. As was illustrated most dramatically by the Stanford team when they








laws say a human must be in control of Google autonomous car driving in the Bay area.

42 Vision for Service Robots Current and Emerging Technologies, Applications, and Markets

Vision systems Design

At present, the military aspect of the unmanned aircraft market has experienced

the highest growth. The civilian markets are sure to follow once the Great Recession of

2009–2010 is past and new budgets allow such activity. To succeed in the military market,

vendors must be aligned with a major defense company and be able to custom-design vision

systems, primarily for this market. One good source of knowledge about the opportunities

is the Association for Unmanned Vehicle Systems International (AUVSI), which sponsors

conferences, trade shows, regional activity chapters, and business opportunities to support

the demand for unmanned aircraft and drones.

Vision Technologies—A UAV’s vision sensors typically include visible, near-infrared, and

infrared cameras, while ultraviolet may be used in limited cases. The vision components

to be utilized in this class of robots are the best in class in terms of speed, resolution, and

communication bandwidth. The combinations are expected to be light, densely packed, and

extremely capable and rugged. Military budgets for these missions range in the millions of

dollars. The vision subsystems are expected to be priced from $100,000 to $500,000.

Sensor fusion in vision systems for UAVs combines real-time and historical records so

observers can tell if something has changed—this is the essence of “persistent stare.” These

systems must work with positioning systems such as GPS and ground-based systems.

Multispectral imaging is important because many things cannot be observed in black and

white; a lot of valuable data are thermal or infrared in nature. All nonvisible light must be

transposed into a vision platform for display, i.e., fused onto a screen. So the vision systems must

have boards that can fuse multiple sensory input from multiple sources to detect things on the

ground such as forests, groves, and bushes and be able to distinguish temperatures of various

objects during different parts of a day, e.g., among humans, land, foliage, rocks, and water.

Market Discussion—Estimates of the market size for UAVs vary considerably, but the

consensus is that the US leads with 60–70% of the world market. Northrop Grumman and

General Atomics are the dominant manufacturers, while Israeli and European manufactur-

ers form a second tier due to lower local investments, and the fact that the governments of

those nations have initiatives to acquire US systems with their higher levels of capability.

The 2010 global UAV market was approximately $5 billion, rising to $11.5 billion in 2020,

according to the Teal Group (Fairfax, VA, USA; www.tealgroup.com).

The UAV vision market is already active with more participants entering in 2011. There

are many offerings and the market is ripe for winnowing, consolidation, and alliances. For

example, one fast growing market is border security—the Market Intel Group (Colorado

Springs, CO, USA; www.marketintelgroup.com) estimates that global market to be $260

million in 2010, rising to $575 million by 2015.

Such estimates are qualified because of the uncertainty in the degree of access UAVs will

have to commercial airspace. Nonetheless, commercial applications promise to soon generate

an order-of-magnitude more economic activity than the now-dominant military UAVs.

4.3.2 Aircraft Support

This area of service robotics pertains to the use of service robots to support manned aircraft.

Although robots have been used for many years for the manufacture of aircraft (notably

for drilling and riveting large sections of an airplane with precision), new applications

are emerging for the use of robots in a service capacity related to operating aircraft. The

To succeed in

the military market,

vendors must be aligned

with a major

defense company

and be able to

custom-design

vision systems.





Although robots have been used for many years for the manufacture of aircraft (notably Such estimates are qualified because of the uncertainty in the degree of access UAVs will



This area of service robotics pertains to the use of service robots to support manned aircraft. million in 2010, rising to $575 million by 2015.









an order-of-magnitude more economic activity than the now-dominant military UAVs.example, one fast growing market is border security—the Market Intel Group (Colorado





an order-of-magnitude more economic activity than the now-dominant military UAVs.are many offerings and the market is ripe for winnowing, consolidation, and alliances. For













have to commercial airspace. Nonetheless, commercial applications promise to soon generate according to the Teal Group (Fairfax, VA, USA; www.tealgroup.com).




test, and evaluation stages. Vision has an important role to play both on the surface and

underwater, in both cases combining cameras and image processing with multiple other

sensors and techniques such as radar, sonar, inertial navigation, GPS navigation (surface

only), and simultaneous localization and mapping (SLAM).

Defense-related remotely operated vehicles (ROVs) and autonomous underwater vehicles

(AUVs) make up approximately 30% of the total unmanned underwater vehicle (UUV)

market, according to naval market analysis firm AMI International (Bremerton, WA, USA).

The strongest growth will be in AUVs for mine countermeasures, followed by port and

harbor security. The next important development will be to “weaponize” the AUVs. Some of

the numbers below assume a 5%/yr growth; AMI expects the military UUV market to grow

at 3–10%/yr to 2015.

Commercial—Offshore energy exploration is the primary driver of developments in

commercial UUVs, with a global 70/30% split between commercial/military vehicle sales.

Spending waxes and wanes with energy prices. According to AMI International, trends in

both the military and commercial UUVs will be for deeper diving, modular payloads, and

multimission capabilities.ROVs and AUVs are typically categorized as large (for commercial, military, oil and gas),

medium (for research, military, oil and gas), and small (for research, military). A medium-

size work-class ROV may have a vision system fielding three color cameras of various types,

two monochrome cameras, and a range of underwater lighting including HID and halogen

lights. An AUV may have at least two cameras.

Scientific—Applications for UUVs in underwater scientific research are numerous,

including research on marine life, seabed formations, and underwater currents and rivers,

and for building subsea research sites. Researchers frequently adapt or use commercially

available small or medium UUVs. On other hand, UUVs developed for research at places

Water: R30.bb Commercial

Vision Units Sold to end of 2009

Vision Units Forecast 2010–2013

Cost per Vision System



ROVs: offshore oil and gas drilling, construction, inspection, mainte-nance, repair; salvage and retrieval (for work after identification); cleaning and containment of spills

5775* 1244*

<$92,500(for large, medium,

small RoVs)

<$115m

immediate through leading UUV suppliers, e.g., schilling Robotics, Kongsberg, Hafmynd, sub-Atlantic

AUVs: offshore oil and gas inspection, maintenance; salvage and retrieval (for identification); inspection of aqueducts, sewers

315* 70* $49,500 $3.47m

immediate through leading UUV suppliers, e.g., schilling Robotics, Kongsberg, Hafmynd, Lockheed martin

*Derived from Ami international estimates

18 Vision for Service Robots Current and Emerging Technologies, Applications, and Markets










—Offshore energy exploration is the primary driver of developments in




ROVs and AUVs are typically categorized as large (for commercial, military, oil and gas),




—Applications for UUVs in underwater scientific research are numerous,




Market Size 2010–2013


immediate through leading UUV suppliers, e.g., Kongsberg, Hafmynd, sub-Atlantic

ileading UUV suppliers, e.g., Kongsberg, Hafmynd, Lockheed

vision systems.

vision systems.18 Vision for Service Robots Current and Emerging Technologies, Applications, and Markets



















—Applications for UUVs in underwater scientific research are numerous,




Market Size 2010–2013

<$115m

$3.47m

custom-design

custom-design

vision systems.




















<$115

$3.47












Adil Shafi and

Conard Holton

Published by

Current and

Emerging

Technologies,

Applications,

and Markets

Vision

for Service

Robots

research

6

to order the Vision for service Robots report • Go to http://qmags.com/VSR to order the full report

or http://qmags.com/VSRE to order the Global Forecast excerpt

• Enter your credit card information and purchase the report online

• A single-use PDF will be delivered to the ordering computer by email

• Cost for the full report is $1,995; cost for the global forecast section only is $995

• For help ordering the report, contact Debbie Bouley at [email protected].

page 18:“Water applications—Commercial Offshore energy exploration is the primary driver of developments in commercial UUVs, with a global 70/30% split between commercial/military vehicle sales ....”

page 42:“Market Discussion—Estimates of the market size for UAVs vary considerably, but the consensus is that the US leads with 60–70% of the world market. Northrop Grumman and General Atomics are the dominant manufacturers, while....”

page 61:“autonomous cars—True driverless vehicles are able to process information and respond like a human driver, and possess the ability to make decisions in real time, including obeying traf-fic laws, avoiding obstacles and other vehicles, and planning and navigating traffic routes. Researchers at VisLab, a spin-off company of the University of Parma, Italy that specializes in vehicular applications involving both environmental perception and intelligent control, developed....”

page 97:“the vision systems for ROVs can be extensive —for example, Eastar Offshore (Singapore) is equipping its latest ROVs for the oil and gas industry with this suite of cameras developed by Kongsberg Maritime (Kongsberg, Norway) ....”

page 103:“competitions—intelligent Ground Vehicles competition Competitions for ground military efforts, held at Oakland University each year; done amongst about 50 universities each year....”

120 Pages of Essential Market Research


http://qmags.com/VSRE

mailto:debbieb%40pennwell.com?subject=

7 For more info: http://www.vision-systems.com/research-reports.html

Frequently asked QuestionsQ: Why is the Vision for Service Robots report an essential

tool for researching vision market opportunities? Multiple near-term market opportunities exist for vision components and systems, totaling several billion dollars. Opportunities in the longer term promise to be even greater. Many countries and regions are avidly fostering research and development in this field. They are doing so because, while industrial robots are sold in the tens of thousands of units per year, service robots will offer a volume of unit sales ultimately in the millions. Many of the companies, university labs, research centers, and govern-ment organizations developing service robots believe that machine vision will provide a critical technical capability and market advantage. With the exceptions of many toys and simple domestic robots such as vacuuming robots, the majority of service robots require machine vision with varied levels of sophistication. Vision components, subsystems, and software range from simple 2-D CMOS sensors to complex subsystems capable of 3-D imaging and pattern recognition. The vision software needed to understand these images and provide robotic feedback is complex, maturing, and evolving rapidly.

Q: Can I find similar research online from other sources? The proprietary research and interviews conducted to create the Vision for Service Robots report are not readily available online. The information available online and through other reports is not sufficient to estimate the size of vision opportunities at the level of detail provided by this report. The report also includes a thorough listing of online resources, including descriptions of several organizations’ service robot projects, which can be used by readers to conduct further research into specific opportu-nities on their own.

Q: What kind of information is included in the Vision for

Service Robots report? The Vision for Service Robots report is based on eight months of interviews and extensive market research resulting in detailed and specific application descriptions of 17 markets and 53 sub-markets for vision systems and components in service robotics. The report divides the applications into air-, land- and water-based categories. For each category, the report includes tables estimating

the current value of installed vision systems, estimated product and component pricing for each application, and the forecasted number and value of vision systems for 2010-2013 and beyond. In addition, the report includes a detailed directory of 100+ organizations involved in service robots—with overviews of their projects and website links—organized by academia, competitions, cor-porations, government/NGOs/researchers, and publica-tions/reports.

Q: Who is the audience for the Vision for Service

Robots report? This report provides a comprehensive review of the technologies, markets, and opportunities for suppliers of vision components, vision subsystem integrators, technol-ogy researchers, universities looking to commercialize their intellectual property, service robot manufacturers, subsystem OEMs, turnkey solution integrators, the finan-cial investment community, and companies and organiza-tions interested in entering up to 17 different service robot market segments.

Q: How long is the Vision for Service Robots report

and in what format is it delivered? The report is 120 pages and includes an executive summary, introduction (with scope and methodol-ogy), global forecast for vision in service robots, critical technologies and definitions, vision for service robots in aerospace applications, vision for service robots in land applications, vision for service robots in water applications, and a resources section with profiles and links to more than 100 organizations involved in service robots. The report is delivered via email to the ordering computer in PDF format. It includes hundreds of links that allow you to find additional information when reading the report from a computer with a live Internet connection.

Q: Who do I contact for more information? To learn more about the contents of the report and how touse the information in it to enter the service robotics fieldfor commercial benefit, please contact Adil Shafi at [email protected]. For help ordering the report, contact Debbie Bouley at [email protected].


EssEntial REsEaRch: Significant Revenue Opportunities for

Documents

Transcript of EssEntial REsEaRch: Significant Revenue Opportunities for