RAM_20130901_Sep_2013

Contents | Zoom in | Zoom out Search Issue | Next PageFor navigation instructions please click here

Contents | Zoom in | Zoom out Search Issue | Next PageFor navigation instructions please click here

___________________

http://www.qmags.com/clickthrough.asp?url=http://www.ieee-ras.org/ram&id=18410&adid=PCOVER 1E1

Pioneer LX

The Pioneer LX is everything you need to get your research rolling.For over 15 years Adept MobileRobots has served the academic

community with intelligent, rugged, research robots. We are proud to

offer our newest robot, the Pioneer LX. Based on the Adept Lynx AIV

(Autonomous Indoor Vehicle) the Pioneer LX combines the

reliable operation of an industrial platform with the open

access and development tools of a Pioneer research robot.

As with other MobileRobots research platforms, developers can

easily access and program the robot‘s on-board computer.

Software libraries are provided to speed the development of

custom applications, and accessible I/O and power supplies make

hardware integration easy. With included laser guidance and auto-

nomous navigation software, the Pioneer LX can be tasked with up to

13 hours of autonomous navigation, carrying up to 60 Kg of sensors,

effectors, or custom accessories.

603.881.7960 www.mobilerobots.com [email protected]

Meet the new State of the Art in Intelligent Research Platforms

Integrated On-Board ComputerIntel D252 64-bit Dual Core 1.8 GHz Atom

2GB DDR3-1066 RAM

Available with Windows or Linux OS

16 In / 16 Out Digital I/O 4 In / 4 Out Analog I/O

Pioneer LX includes:Autonomous Navigation and Mapping Software SICK S300 Laser Scanner Wireless Ethernet CommunicationColor LED Status Indicator RingsAccessory Mounting Deck (Removable) Docking station for Autonomous or Manual ChargingSpeakers & Voice Synthesis Software

Pioneer Software Development Kit

Previous Page | Contents | Zoom in | Zoom out | Front Cover | Search Issue | Next PageIEEE

M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM

Qmags®THE WORLD’S NEWSSTAND

qqM

Mq

qM

MqM


________________________________

http://www.qmags.com/clickthrough.asp?url=www.mobilerobots.com&id=18410&adid=PCOVER 2A1

mailto:[email protected]

http://www.qmags.com/clickthrough.asp?url=www.ieee-ras.org/ram&id=18410&adid=logo

http://www.qmags.com/clickthrough.asp?url=www.qmags.com&id=18410&adid=logo



1SEPTEMBER 2013 IEEE ROBOTICS & AUTOMATION MAGAZINE

Vol. 20, No. 3 SEPTEMBER 2013ISSN 1070-9932http://www.ieee-ras.org/publications/ram

50 Robotized Plant Probing Leaf Segmentation Utilizing Time-of-Flight Data By Guillem Alenyà, Babette Dellen, Sergi Foix, and Carme Torras

60 Structure-Reconfiguring Robots Autonomous Truss Reconfiguration and Manipulation

By Franz Nigl, Shuguang Li, Jeremy E. Blum, and Hod Lipson

72 Development of the UB Hand IV Overview of Design Solutions and Enabling Technologies

By Claudio Melchiorri, Gianluca Palli, Giovanni Berselli, and Gabriele Vassura

82 All the Robots Merely Players History of Player and Stage Software

By Geoffrey Biggs, Radu Bogdan Rusu, Toby Collett, Brian Gerkey, and Richard Vaughan

Digital Object Identifier 10.1109/MRA.2013.2272200

FEATURES

26 Autonomous Underwater BiorobotsA Wireless System for Power Transfer

By Tareq Assaf, Cesare Stefanini, and Paolo Dario

33 Tracking Aquatic Invaders Autonomous Robots for Monitoring Invasive Fish By Pratap Tokekar, Elliot Branson, Joshua Vander Hook,

and Volkan Isler

42 Under the Sea Rapid Characterization of Restricted Marine Environments By Alberto Alvarez, Jacopo Chiggiato, and Baptiste Mourre

IMA

GE

LIC

EN

SE

D B

Y IN

GR

AM

PU

BLI

SH

ING

If you like an article, click this icon to record your opinion. This capability is available for online Web brows-

ers and offline PDF reading on a con-nected device.

ON THE COVERWaterproof, contactless solutions for powering underwater robots have been developed in the Lampetra Project (www.lampetra.eu) at The Biorobotics Institute of the Scuola Superiore Sant’Anna, Pisa (Italy).

PHOTO COURTESY OF TAREQ ASSAF, CESARE STEFANINI, AND PAOLO DARIO


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=http://www.ieee-ras.org/publications/ram&id=18410&adid=P1E2

http://www.qmags.com/clickthrough.asp?url=www.lampetra.eu&id=18410&adid=P1E1





2 IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2013

4 FROM THE EDITOR’S DESK

6 PRESIDENT’S MESSAGE

12 COMPETITIONS

16 INDUSTRIAL ACTIVITIES

22 TC SPOTLIGHT

92 STUDENT’S CORNER

94 WOMEN IN ENGINEERING

96 ON THE SHELF

98 SOCIETY NEWS

103 CALENDAR

A Publication of the IEEE ROBOTICS AND AUTOMATION SOCIETYVolume 20, No. 3 September 2013 ISSN 1070-9932 http://www.ieee-ras.org/publications/ram

COLUMNS & DEPARTMENTS

IEEE Robotics & Automation Magazine (ISSN 1070-9932) (IRAMEB) is published quarterly by the Institute of Electrical and Electronics Engineers, Inc. Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997 USA, Telephone: +1 212 419 7900. Responsibility for the content rests upon the authors and not upon the IEEE, the Society or its members. IEEE Service Center (for orders, subscriptions, address changes): 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855 USA. Telephone: +1 732 981 0060. Individual copies: IEEE members $20.00 (first copy only), non-members US$110.00 per copy. Subscription rates: Annual subscription rates included in IEEE Robotics and Automation Society member dues. Subscription rates available on request. Copyright and reprint permission: Abstracting is permitted with credit to the source. Libraries are permitted to

photocopy beyond the limits of U.S. Copyright law for the private use of patrons 1) those post-1977 articles that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA; 2) pre-1978 articles without a fee. For other copying, reprint, or republication permission, write Copyrights and Permissions Department, IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854. Copyright @ 2013 by the Institute of Electrical and Electronics Engineers Inc. All rights reserved. Periodicals postage paid at New York and additional mailing offices. Postmaster: Send address changes to IEEE Robotics & Automation Magazine, IEEE, 445 Hoes Lane, Piscataway, NJ 08854 USA. Canadian GST #125634188 PRINTED IN THE U.S.A.

IEEE prohibits discrimination, harassment, and bullying. For more information,visit http://www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

EDITORIAL BOARDEditor-in-ChiefEugenio Guglielmelli ([email protected])Campus Bio-Medico University, Roma (Italy)

Associate EditorsRaffaella CarloniUniversity of Twente(The Netherlands)

Antonio FranchiMax Planck Institute for Biological Cybernetics (Germany)

You-Fu LiCity University of Hong Kong (SAR China)

Yi GuoStevens Institute of Technology (USA)

Yu SunUniversity of South Florida (USA)

Loredana ZolloCampus Bio-Medico University, Roma (Italy)

Past Editor-in-ChiefPeter CorkeQueensland University of TechnologyBrisbane, Australia

Industry EditorRaj MadhavanUniversity of Maryland College Park (USA)

Video EditorJonathan RobertsCSIRO (Australia)

COLUMNSCompetitions: Stephen Balakirsky

From the Editor’s Desk: Eugenio Guglielmelli

ROS Topics: Steve CousinsWillow Garage (USA)

On the Shelf: Alex SimpkinsRDP Robotics (USA)

Student Corner: Laura MargheriThe BioRobotics Institute (Italy)

IEEE RAS Vice-Presidentfor PublicationsAlessandro De LucaUniv. of Roma “La Sapienza” (Italy)

RAM homepage:http://www.ieee-ras.org/publications/ram

Robotics and AutomationSociety Project SpecialistsKathy ColabaughRachel O. [email protected]

Advertising SalesSusan SchneidermanBusiness Development ManagerTel: +1 732 562 3946Fax: +1 732 981 [email protected]

IEEE PeriodicalsMagazines DepartmentDebby NowickiManaging [email protected]

Janet DudarSenior Art DirectorGail A. SchnitzerAssistant Art Director

Theresa L. SmithProduction Coordinator

Felicia SpagnoliAdvertising Production Manager

Peter M. TuohyProduction Director

Dawn M. MelleyEditorial Director

Fran ZappullaStaff Director,Publishing Operations

IEEE-RAS Membershipand Subscription Information:+1 800 678 IEEE (4333)Fax: +1 732 463 3657http://www.ieee.org/membership_services/membership/societies/ras.html



M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________________

_____

_______

_____

_________

________

http://www.qmags.com/clickthrough.asp?url=http://www.ieee.org/web/aboutus/whatis/policies/p9-26.html&id=18410&adid=P2E4

http://www.qmags.com/clickthrough.asp?url=http://www.ieee.org/membership_services/membership/societies/ras.html&id=18410&adid=P2E3



http://www.qmags.com/clickthrough.asp?url=www.sfiprogram.org&id=18410&adid=P2E5









www.youBot-Store.com

Mobile manipulation for research and education

LATEST YOUBOT NEWS:

KUKA receives overwhelming response to its 20.000 EUR KUKA Innovation in

Mobile Manipulation Award

Best student project sought. Winner will receive up to 1.500 EUR

Available in late summer: From simulation to implementation with

KUKA youBot, NI LabVIEW and NI CompactRIO

youBot Store grants up to 20% volume discount to purchaser consortia

KEY FEATURES

omnidirectional mobile platform

5-DOF manipulator

two-fi nger gripper

real-time EtherCAT communication

open interfaces

open source repository

arm and platform separately or

confi gurations of arm and platform

youBot

KUKA youBot 5-DOF arm with 2-fi nger gripper and wrist camera

KUKA youBot omni-directional mobile platform with arm

KUKA youBot “on tour” KUKA youBot RoboCup@WorkConfi guration B

KUKA youBot omni-directional mobile platform with dual arm


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.youBot-Store.com&id=18410&adid=P3A1





FROM THE EDITOR’S DESK

4 IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2013 1070-9932/13/$31.00©2013IEEE

Your MagazineBy Eugenio Guglielmelli

It is a real privilege and a terrific opportunity for me to start my term as IEEE Robotics and Automation Magazine (RAM) editor-in-chief with

this September issue. I will serve for five years, and throughout my term, I will always rely on your opinions, criticisms, suggestions, and any other inputs that could help me to further improve your level of satisfaction as contributors and readers of the magazine.

This is the IEEE Robotics and Automation Society’s (RAS’s)official magazine, which means that, as RAS members, it is your maga-zine—an impor-tant service that the Society pro-vides to all of you, and to which you can help shape to fulfill your actual needs of commu-nicat ing your research, prod-ucts, and events within and outside our community.

My main per-sonal commit-

ment and highest priority is to guarantee the fastest possible submission-to-publication time, as compatible with the quality of the review process that is required by a top robotics and automation publication like RAM.

As you know, RAM guarantees rapid posting of all papers that have successfully passed the peer review process, upon completion of an addi-tional, dedicated editing effort. Each paper, in fact, is fully edited by our pro-fessional team to give your presenta-tion that special touch that you can easily and immediately perceive in any of our articles. RAM also devotes up to three of our four issues per year to Spe-cial Issues/Focused Sections. This means that only one issue per year is fully devoted to regular submissions, which are a significant portion of our total submissions because of the good reputation of this journal in the robot-ics and automation field.

In other words, when submitting an article to RAM, you may have to wait a few months longer than you would for other publications to see your work printed, but the high quality of the final presentation, the unique opportunity to directly share your achievements with all of the members of our RAS commu-nity, and the top-level ranking of RAMwill definitely reward your patience. In addition, as with all other IEEE publica-tions, you have the possibility to select

the Open Access option when submitting your articles. This will make the time needed for rapid

posting of your papers exactly the same as the time

needed for completing the Open Access publication process.

I will mainly use this column to keep you up to date with the latest ongoing initiatives promoted by this magazine. I believe that RAM is the ideal laboratory and incubator of innovations for RAS publications: it has been so in the past, and I would love to keep the same level of quality as my great predecessors. I believe Ste-fano Stramigioli and Peter Corke, with whom I had the privilege to work as an associate editor of RAM, did an incredible job in amplifying the impact and shaping the contents of this magazine, fully inline with its scope. Real-world robotics papers, high-quality tutorials and surveys, special issues on emerging topics, and reports on the latest achievements are the typical contributions sought for publication in RAM. In addition, I am already discussing some new ideas with the editorial board and the IEEE to keep this magazine at the forefront of both robotics and automation research and publication services. You will hear about my ideas in the near future for sure!

Real-world robotics

papers, high-quality

tutorials and surveys,

special issues on

emerging topics,

and reports

on the latest

achievements

are the typical

contributions

sought for

publication

in RAM.

Digital Object Identifier 10.1109/MRA.2013.2275691Date of publication: 11 September 2013


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM






http://www.qmags.com/clickthrough.asp?url=http://www.qmags.com/ram/like/?RAS Journal Publications Provide an Open Access Option&id=18410&adid=P4E1


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________________________

http://www.qmags.com/clickthrough.asp?url=www.cyberbotics.com&id=18410&adid=P5A1





PRESIDENT’S MESSAGE


Robotics and Automation for Humanitarian Activities

By David E. Orin

The IEEE International Confer-ence on Robotics and Auto-mation (ICRA) was held in Karlsruhe, Germany, 6–10

May 2013. This was the first time that ICRA was held in Germany, which has the largest IEEE Robotics and Auto-mation Society (RAS) membership in Europe, and the technical and social programs were outstanding. I would like to thank General Chair Rüdiger Dillmann, Program Chair Markus Vincze, Local Arrangements Chair Tamim Asfour, and the organizational team for a truly memorable confer-ence in the heart of Europe.

This year, we were pleased to have IEEE President Peter Staecker as a guest. At the awards ceremony, he presented the Robotics and Automation Technical Field Award and participated in many other RAS activities, including the Stu-dent’s Lunch with Leaders and the IEEE Graduates of the Last Decade (GOLD) Lunch. President Staecker and I are shown in Figure 1 with past presidents of RAS and other IEEE Society and Council presidents.

The Administrative Committee of RAS held its first meeting in 2013 at ICRA and approved many new pro-grams. Among these were two pro-grams, with a total funding of nearly US$25,000, to support humanitarian activities in robotics and automation (R&A). We were the first IEEE Soci-ety to form a Special Interest Group in Humanitarian Technology (SIGHT) in 2012, after which our activities continue to grow. The organizer and chair of IEEE RAS SIGHT is Raj Mad-

havan of the University of Maryland, College Park.

The IEEE and RAS are both focused on advancing technology for humanity. While this may involve a broad array of activities, one particular area of interest to many of our officers and members is the area of humanitarian technology: how R&A technologies can make a sig-nificant difference in raising the quality of life for humanity. This includes a strong interest in technologies that can have an impact in underserved, under-developed regions of the world.

The first major program for IEEE RAS SIGHT was to support an educa-tional activity with the African Robotics Network (AFRON). AFRON is a com-munity of institutions, organizations, and individuals formed to support robotics research and education in Africa (www.robotics-africa.org). It was cofounded by Ayorkor Korsah of Ashesi University, Ghana, and Ken Goldberg of the University of California, Berkeley. IEEE RAS SIGHT funded the prizes for

AFRON’s “10-Dollar Robot Design Challenge” to design an educational robot using less than US$10 for parts. The challenge was meant to get students interested in robotics and, more broadly, in engineering. We funded prizes for ten very creative designs. The winner in the tethered category is based on surplus Sony PlayStation game controllers (http://spectrum.ieee.org/automaton/robotics/diy/winners-of-10-robot-challenge-announced). At US$8.96 in parts, it indeed came in with a cost under US$10.

In 2013, IEEE RAS SIGHT is sup-porting a new AFRON robotics com-petition with the aim of developing an ultra-affordable educational robot to collaboratively design robots that can inspire young children worldwide about science, technology, engineer-ing, and math. The project engages interested students and engineers, in Africa and worldwide, in designing robots that are an order of magnitude less expensive than existing products.


Figure 1. IEEE President Peter Staecker and IEEE RAS President David Orin with past presidents of RAS and other IEEE Society/Council presidents at ICRA 2013 in Karlsruhe, Germany. From left, Ning Xi (past president, Nanotechnology Council), Kazuhiro Kosuge (junior past president, RAS), Peter Staecker, T. J. Tarn (past president, RAS), Vladimir Lumelsky (president, Sensors Council), David Orin, Christoph Stiller (president, Intelligent Transportation Systems Society), Ren Luo (past president, Industrial Electronics Society), Bruno Siciliano (senior past president, RAS), Toshio Fukuda (past president, RAS), Paolo Dario (past president, RAS), and Raja Chatila (president-elect, RAS).


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_________________________

______________

http://www.qmags.com/clickthrough.asp?url=http://spectrum.ieee.org/automaton/robotics/diy/winners-of-10-robot-challenge-announced&id=18410&adid=P6E2

http://www.qmags.com/clickthrough.asp?url=www.robotics-africa.org&id=18410&adid=P6E1





Hunting For High Quality Gears?

Forest City Gear Leads the Way

11715 Main Street, Roscoe, IL 61073815-623-2168

US Navy MH-60S Seahawk helicopters patrolling for mines out ahead of surface vessels use a

powerful and precise Carriage Stream Tow and Recovery System (CSTRS) to quickly raise and

lower mine-hunting and destruction equipment. Very high-precision gears from Forest City Gear

help to ensure that the mission goes as planned. In this and many other aerospace and defense

applications, Forest City Gear is helping customers meet their gear challenges – no matter how

difficult to detect.

Don’t let gear challenges go undetected. Visit www.forestcitygear.com.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.forestcitygear.com&id=18410&adid=P7A1






M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________________________________________

__________

_________

http://www.qmags.com/clickthrough.asp?url=www.servocity.com&id=18410&adid=P9A1


http://www.qmags.com/clickthrough.asp?url=www.servocity.com&id=18410&adid=P9A2






With RAS support, AFRON is cur-rently developing the second phase of the project that includes an interna-tional challenge in 2013 with three categories: software, lesson plans, and hardware design improvements to the winning robot designs (making them more functional, more reliable, easier to use, less expensive, and easier to manufacture).

IEEE RAS SIGHT has many other plans, and the funds for 2013 are tar-geted to pursue other educational, industrial, and technical partnerships. In 2012, Madhavan organized a spe-cial session at the IEEE Global Humanitarian Technology Conference (GHTC) in Seattle, Washington. It was a good opportunity for participants to share their experiences with using R&A technologies for humanitarian activities, such as the rescue robotics work of Robin Murphy, the director of the Center for Robot-Assisted Search

and Rescue at Texas A&M University. Plans are being made to increase our support for GHTC 2013 in Silicon Valley, California.

IEEE RAS SIGHT is also organizing a Humanitarian R&A Technology Chal-lenge at ICRA 2014 in Hong Kong. The focus of the challenge is on landmine clearance. This is also the focus of a com-petition of our RAS Chapter in Egypt, which is organizing the first international outdoor robotics competition on humanitarian demining in September in Cairo (www.landminefree.org).

John Muir, the famous naturalist once said: “When we try to pick out anything by itself, we find that it is bound fast by a thousand invisible cords that cannot be broken, to every-thing in the universe.” At the heart of our humanitarian work in RAS is an understanding that we are all con-nected in this world, even in ways that we do not yet fully appreciate. Our

humanitarian activities can give us a more direct way to respond across societal bounds to use our technolo-gies to enhance life globally, especially for the most vulnerable in the under-served and underdeveloped parts of the world.

If you are interested in learning more about IEEE RAS SIGHT, and applying R&A technologies for humanitarian causes, feel free to contact Raj Madha-van at [email protected]. AFRON also welcomes new members. See their Web site to join: www.robotics-africa.org. I would encourage you to provide some small measure of your time and expertise to humanitarian causes. Hope-fully, with more of our members volun-teering in this work, we will engage in an increasing set of activities that can have a positive impact on improving the quality of life in our world.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_________________

_________________

__

______________

http://www.qmags.com/clickthrough.asp?url=www.robotics-africa.org&id=18410&adid=P10E2

http://www.qmags.com/clickthrough.asp?url=www.landminefree.org&id=18410&adid=P10E1


http://www.qmags.com/clickthrough.asp?url=www.robotnik.eu&id=18410&adid=P10A1







6 degrees of freedom

Powerball Lightweight Arm LWA 4P

340° rotation range

Powerball ERB

76 measuring points per detection field

Servo-electric3-Finger-Hand SDH

Jens Lehmann, Brand Ambassador of SCHUNK, the family-owned company

Mobile Gripping Systems.It’s time to make use of your machine’s full potential.www.us.schunk.com/machine-potential


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.us.schunk.com/machine-potential&id=18410&adid=P11A1





COMPETITIONS


Upgrade Your Robot Competition, Make a Festival!

By Fanny Riedo, Mariza Freire, Julia Fink, Guillaume Ruiz, Farinaz Fassa, and Francesco Mondada

In 1992, Jean-Daniel Nicoud started the first edition of an internal robot-ics contest at the École Polytechnique Fédérale de Lausanne, Switzerland.

In 2008, Francesco Mondada opened it up to the broader public and launched the robotics festival, which comprised workshops, exhibitions, and events in addition to the competition (Figure 1). Today, the robotics festival is one of the largest events in Europe, attracting 17,000 visitors on a single day. The goal of the robotics festival is to foster a bet-ter understanding of technology and robotics in our society and to do this in an open way. In this article, we outline why we think the image given by robot-ics competitions is restrictive and what people’s motivations are to attend these events, particularly girls and women, who are clearly underrepresented. We conclude with suggestions for design-ing and advertising such activities in the future.

Robotics is playing an increasingly important role in our daily lives, but most people are not educated in how to deal with this kind of autonomous tech-nology. This can lead to strong negative or positive overreactions. For instance, a large-scale survey in Europe [1] revealed that, despite a generally positive view on robots, the majority of European citizens (60%) felt that robots “should be banned” from tasks such as helping dis-abled people or caring for elderly and children, which contrasts with the trend

in research to improve quality of life with assistive robotics.

This lack of educa-tion on robotics also generates inequalities and gaps between sub-groups in society. For instance, women, who are clearly underrepre-sented in technology-related fields, tend to have a worse image of robotics than men [1]. This gender imbalance is detrimental to the field, as studies have shown that collective intelligence depends on the proportion of females in the group [2].

The gender imbalance is reflected in robotics competitions. In Switzerland, one of the most popular robotics com-

petitions is the First Lego League. Among its 660 participants in 2012, only 20% were girls. The EuroBot con-test, the largest contest for technical schools, scored worse, with young women representing less than 5% of the nearly 100 participants. The Swiss


Figure 1. The robot competitions are a core event of the robotics festival, held 20 April 2013.

Figure 2. The 57 robot exhibitions, including the one shown here of a flying robot in action, were very attractive and generated many exchanges between public and specialists.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM






While the world benefi ts from what’s new,IEEE can focus you on what’s next.

Request a Free Trialwww.ieee.org/tryieeexplore

Develop for tomorrow with today’s most-cited research.

Over 3 million full-text technical documents

can power your R&D and speed time to market.

IEEE Xplore® Digital LibraryDiscover a smarter research experience.

Follow IEEE Xplore on


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.ieee.org/tryieeexplore&id=18410&adid=P13A1






national Bug and Play contest reached 30% females among the 259 partici-pants. This contest revolves around technology but mainly promotes its artistic and creative side.

With our festival, our aim was to promote related activities in a nondis-criminatory manner (Figure 2). In 2013, we had an average of 27% female participation among the 2,300 participants in workshops. Some activities even reached a proportion of 67% girls. Several surveys conducted during the festival helped us under-stand where these participation differ-ences come from.

The first clear aspect we could observe is that the proportion of girls and boys differs by their age group (Figure 3). Both in 2012 and 2013, the mean age of boys taking part in the workshop was significantly higher than that of girls. Concretely, we observed that, at a young age, the number of girls and boys is comparable, but for older children, the proportion of boys increases. These observations are coher-ent with some sociological assumptions stating that interest in scientific fields or professions at the age of the con-struction of sexual identity is driven by the image that these fields or profes-sions diffuse [3]. For instance, boys would turn to sciences and technology because many men are already involved in this field, whereas girls would show interest in humanities because in that field women outnumber men.

In the robotics festival, differences in motivations between genders were discovered through two surveys car-ried out among more than 1,300 par-ticipants over two years. Whereas most men and boys stated that they joined the event because they are interested in technology (65%), only 40% of women and girls attended because of their interest in technology, 25% joined out of interest in the social event and 22% because of the interest in technology of the men in their family. Despite a less personal initial reason to attend the festival, girls and women participated in the workshops and events and appreciated them, showing that they are not reluctant to participate in these types of activities (Figure 4).

The angle from which activities are advertised could be a determinant, as hinted at by the success of the Bug and Play contest, which promotes its cre-ative side. However, we could not make a clear link between the workshop top-ics and their female attendance. What we did observe is that some workshop descriptions that were written by men and included masculine terminology [3] attracted very few girls. We also noted that the workshops with the highest female attendance were often organized by women.

In 2012, we sought to highlight the effect of activity advertisement by dis-tributing 1,500 free robotics kits associ-ated with a choice. The basic kit, which comprised a set of parts to build a robot figurine, was the same for everyone, but two possible activities were sug-gested: a collaborative, creative one or a competitive, game-oriented one. The participants would choose one activity and then receive the corresponding accessories. We recorded the gender and the choice of activity of 1,428 par-ticipants. This choice does not seem to be influenced by gender, as the propor-tion of females was similar in both cate-gories (38 and 35%). However, even for males, collaboration is the preferred option, as two thirds of all participants opted for this one.

In conclusion, the image of robotics competitions seems to be attractive only to a specific segment of the pub-

lic, leaving aside an important part of the audience. This is especially obvious with women and girls. Thus, although competitions are a precious tool to educate people, they are not sufficient. Our experience shows that having a variety of activities and promoting their different aspects, be it creative, fun, or social, gives a wider impact to the venture, reaching not only more girls but also boys who appreciate the noncompetitive side of robotics. We should also keep in mind that the older the girls are, the more difficult it is to reach them. Finally, the communica-tion of the event is primordial: involv-ing females in the organization and the advertisement of the event is likely to broaden its scope and give a more open image to the audience.

The festival organization and the survey have been supported by the Swiss National Center of Competence in Research “Robotics.”

References[1] TNS Opinion & Social. (2012, Sept.). Public attitudes towards robots: Special Eurobarometer 382 [Online]. Available: http://ec.europa.eu/pub-lic_opinion/archives/ebs/ebs_382_en.pdf[2] A. W. Woolley, C. F. Chabris, A. Pentland, N.Hashmi, and T. W. Malone, “Evidence for a collec-tive intelligence factor in the performance of human groups,” Science, vol. 330, no. 6004, pp. 686–688, 2010.[3] Françoise Vouillot, “L’orientation aux prises avec le genre,” Travail, genre et sociétés, vol. 2, no. 15, pp. 87–108, Nov. 2007.[4] A. Herdağdelen and M. Baroni, “Stereotypical gender actions can be extracted from Web text,” J.Amer. Soc. Inform. Sci. Technol., vol. 62, no. 9, pp. 1741–1749, Sept. 2011.

Figure 4. In one of the 2,250 festival workshops, participants could assemble, solder, or program robots.

Age and Gender Distribution at the Workshops (2012 and 2013)

MF

Gen

der

45 6 7 8 9 10 11 12 1314

Age

0.0

0.2

0.4

0.6

0.8

1.0

Figure 3. The proportion of gender by age of the 1,755 participants in the workshops of the last two editions of the festival, illustrating the increasing proportion of males in respect to their age. The width of the bars is proportional to the number of individuals in each category.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


____________

____________________

http://www.qmags.com/clickthrough.asp?url=http://ec.europa.eu/public_opinion/archives/ebs/ebs_382_en.pdf&id=18410&adid=P14E1







M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.wiley.com/ieee&id=18410&adid=P15A2

http://www.qmags.com/clickthrough.asp?url=www.wiley.com/IEEE&id=18410&adid=P15A1





16

Robotics and Automation Activities in South Africa

By Simukai Utete, Jeremy Green, Ashley Liddiard, and Chris R. Burger

outh Africa sits at the southern tip of Africa. The country has a population of 52.98 million, according to 2013 statistical esti-

mates [1]. South Africa comprises nine provinces, with major cities including Johannesburg, Cape Town, Pretoria, and Durban. Sectors such as mining and agriculture have traditionally been large, but the government is working to move more from a resource-based economy to a knowledge-based economy. The coun-try is involved in major science projects, such as the Square Kilometre Array, and there is a significant research taking place in a range of areas including biol-ogy and medical sciences. Robotics is also gaining importance in South Africa’s science and technology land-scape in industry as well as in academic institutions and science councils. This introduction to robotics and automation in South Africa seeks to give a flavor of some of the developments in the coun-try. It is by no means exhaustive.

The importance and potential of robotics are recognized by South Africa’s Department of Science and Technology (DST). The DST, which has previously supported strategies for nanotechnology and photonics, is now considering stakeholder submis-sions toward the formulation of a national robotics strategy. A strategy

development process involving inter-actions of potential stakeholders has been running for more than two years and has identified areas where robotics could be important for pre-serving jobs and creating new ones, as well as improving productivity and competitiveness [2].

Given South Africa’s history and the need for development to widen the scope of opportunity, the human angle is key. Human capital development would be a thrust of any strategy. This would encompass tertiary study as well as the training of technicians and arti-sans who could maintain robotics infra-structure, allowing for a larger share of demand for maintenance activity to be met from within South Africa.

In general, a discussion of robotics in many contexts raises the issue of employment, the question of whether robots will reduce jobs. There are spe-cific jobs in certain industries where this might be an issue, but robotics can also be a driver for the creation of new jobs, and new types of industry, as well as a force to improve certain types of jobs. Stakeholder engagements around the robotics strategy brought out many advantages, which a robotics focus could confer, including improved com-petitiveness in certain industries and the provision of better working environ-ments for certain types of activity. One example of the latter is the potential for the use of robotics to enhance safety.

The strategy discussion to date has identified the initial focus areas for potential impact in South Africa as mining, flexible manufacturing, and medicine and health care [2].

Mining RoboticsMining robotics is one of South Africa’s biggest opportunities for robotics research and development to promote greater safety, extend the life of mines, and improve competitive-ness. One reason is that there are large reserves of gold in underground deposits, which might become feasi-ble to mine through new technolo-gies, including robotics. Unexploited deposits are located in stability pil-lars, in areas that are unsafe for peo-ple, or found to be nonprofitable for extraction because of low grade or narrow deposit using existing mining technologies [3].

South African gold and platinum deposits share characteristics in that they are narrow (5 cm to 1.5 m) bands of ore hundreds of kilometers wide, dipping into the earth at between 12° and 30° from the surface to as yet uncharted depths. Challenges in min-ing at extreme depths of more than 5 km underground are such that the use of people to mine, as in the tradi-tional narrow stoping, is not feasible. At this depth, the rock stresses make it unsafe for people to be near the rock face because of the high risk of a rock

Second in the series of articles focusing on the state of robotics and automation in the BRICS countries: Brazil, Russia, India, China, and South Africa, this article provides an overview on South Africa written by researchers from the Council for Scientific and Industrial Research. The objective of this series is to inform the readers of the unique challenges

that these countries have faced and the solutions they have adopted to solve their problems and to facilitate discussions with the interested members of the community. Please send your comments and feedback to the IEEE Robotics & Automation Society (RAS) Vice President of the Industrial Activities Board Raj Madhavan at [email protected].


INDUSTRIAL ACTIVITIES

IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2013

S


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


______________







burst, and the environment is extreme, with temperatures reaching higher than 50 °C and almost 100% humidity. This presents significant opportunities in the area of robotics [3]. Robotics activities could unlock reserves that could extend mining operations and enhance safety and productivity. Robotics appli-cations include remote sensing of reef grade and hanging wall stability, auton-omous ore transport, and access to extreme depths, among others.

A number of mining houses have robotics programs. For example, Anglo American recently announced that they will work with Carnegie Mellon University on mining robotics interven-tions aimed at enhancing safety.

Among science councils, the Council for Scientific and Industrial Research (CSIR) undertakes research and development in robotics. For exam-ple, a partnership among different CSIR groups (involving its Centre for Mining

Innovation, Mobile Intelligent Autonomous Systems Field Robotics Group, and Mechatronics and Micro-manufacturing Group) recently pro-duced a prototype mine safety platform, which is aimed at inspection of the hanging wall (ceiling) in underground mines to identify high-risk areas before people are allowed to enter [5].

AutomotiveMuch of the robotics activity in South Africa is currently focused on automa-tion, particularly in the automotive industry. Major automotive plants exist across the country, including Volkswagen in Port Elizabeth and BMW in Pretoria. The potential for and use of robotics is not only in the plants but in supporting industries, which also employ automation and robotics. Although there are large bodyshops, supporting hundreds of robots, relatively few local companies

install and comprehensively maintain robots, a potential area for a national robotics strategy to address. Outside the automotive sector, there is also potential demand from sectors as var-ied as agriculture, pharmaceuticals, and the food and beverage industry.

Human CapitalThe sustainability of a robotics pro-gram depends on the people, including the engineers, researchers, technicians, and artisans who will make it happen. To this end, there have already been many human capital initiatives around robotics. At the tertiary level, one example of a human capital develop-ment program has been activity based around RoboCup [6]. The DST and the National Research Foundation (NRF) set up a program to encourage robotics research. In 2007, Alexander Ferrein, a German researcher involved in robotics research, visited South Africa at the


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________________________

http://www.qmags.com/clickthrough.asp?url=www.lacquey.nl/IEEE&id=18410&adid=P17A1






invitation of the German Embassy in Pretoria. Ferrein was sponsored by the NRF and hosted by the CSIR through its Meraka Institute.

After Ferrein’s visit, CSIR Meraka submitted a proposal to the DST for a multiyear robotics research project in South African universities. The robot-ics research would revolve around par-ticipation in RoboCup’s Small Size League, in which each university would field a team of robots to play soccer according to standardized rules. The league was specifically selected to ensure that each team would have to learn about robot construction, image processing, and decision-making strat-egy, all elements that are important for real-world robotics.

The DST provided funding for a project, resulting in the establishment of four teams: 1) Stellenbosch University, 2) the University of Cape Town, 3) University of KwaZulu Natal, and 4) the University of Pretoria. Activity commenced in the 2008 aca-demic year and terminated in 2012. Over its duration, the project has sup-ported at least five Ph.D. and 17 mas-ter’s students. In addition, many under-graduate students were involved in aspects of the research. Several dozen research publications have resulted directly or indirectly from the research. Pending the completion of reporting, it is estimated that more than 80% of the supported students have graduated.

Research at tertiary institutions around South Africa is varied, with groups focusing on a variety of appli-cations, from ground vehicles to unmanned aircraft systems (UASs). For example, the University of Johan-nesburg has been a key participant in the South African solar challenge with a dual drive vehicle that it aims to drive autonomously using vehicle-fol-lowing technology. Efforts in aerial autonomy have been under way at several universities. With support from the Advanced Manufacturing Technology Strategy, reconfigurable production robotics has been a focus of work at other tertiary institutions.

Future robotics programs would seek to expand robotics education and research, including involving previ-ously disadvantaged universities. The robotics strategy proposals also iden-tify a need to train technicians, arti-sans, and others to create a sustainable base for robotics work in industry. The multidisciplinary nature of robotics also means that developments in this area translate into skills, which can be used in sensor system development, signal processing, mechanical design, and other applications.

The Robotics ConversationIn addition to work directly in robotics, a range of robotics-relevant research also takes place in South Africa, for example, in artificial intelligence [7]. There are also national conferences in robotics and related areas. RobMech (www.robmech.co.za), an annual robot-ics and mechatronics conference, is per-haps the most robotics themed. RobMech aims to bring together indus-try and research and encourage collabo-ration. The conference is supported by organizations such as the South African Chapter of the IEEE Robotics and Automation Society (RAS), and in the past has also been supported by the Technology Innovation Agency.

There is also a well-established annual pattern recognition conference, the Pattern Recognition Association of South Africa Symposium (www.prasa.org). Related international conferences have also been hosted in South Africa, bringing the opportunity for engage-ment with robotics-relevant areas of engineering; for example, the 2014 International Federation of Automatic Control World Congress will take place in Cape Town.

There is significant IEEE activity in South Africa, including in robotics. The South African Chapters of RAS and the Control Systems Society merged in 2009 to form a joint Chapter. The current focus includes supporting RobMech and helping build a robotics community that has more interaction between the vari-ous aspects of robotics.

ConclusionsOne role envisaged in the submissions for a national robotics strategy is an inte-grative one, which would bring together academic, industrial, and other robotics-themed programs to create greater impact by helping to support work across groups focusing on specific areas, such as biomedical applications or flexi-ble manufacturing [2].

Robotics in South Africa is poised at an interesting point, where submissions for a national strategy are being sought and coordinated. Discussions among stake-holders from a range of application areas, and the DST’s interest in considering sub-missions, show a recognition of the impor-tant role robotics can play in promoting social good and economic productivity.

Acknowledgment The authors would like to thank Matlho J. J. Molapisi and Xolani Makhoba, from the South African DST, for discussions on this theme.

References[1] Statistics South Africa. (May 2013). Statistical release P0302—Mid year population estimates[Online]. Available: http://www.statssa.gov.za/publications/P0302/P03022013.pdf[2] N. Tlale, R. Coetzee, I. Craig, R. Kuppuswamy,M. Reis, T. van Niekerk, and D. Waller, “A national robotics strategy for South Africa,” Draft version for discussion, Version 3.0, Nov. 2012.[3] J. Green and D. Vogt, “Robot miner for low grade narrow tabular ore bodies: The potential and the challenge,” in Proc. 3rd Robotics Mechatronics Symp., Pretoria, South Africa, Nov. 8–10, 2009.[4] Anglo American. (June 2013). Anglo American and Carnegie Mellon University collaborate on robotics development to improve safety in its min-ing operations, press release [Online]. Available: http://www.angloamerican.co.za/media/press-releases/2013/pr16-01-2013.aspx[5] J. Green, P. Bosscha, L. Candy, K. Hlophe, S. Coetzee,and S. Brink, “Can a robot improve mine safety?” in Proc. 25th Int. Conf. CAD/CAM, Robotics Factories Future, Pretoria, South Africa, July 13–16, 2010.[6] RoboCup South Africa. (June 2013). [Online]. Available: http://robocup.org.za [7] A. Ferrein and T. Meyer, “A brief overview of artificial intelligence in South Africa,” Artif. Intel. Mag., vol. 33, no. 1, pp. 99-103, Spring 2012.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__

_________________

_________________________

________________

http://www.qmags.com/clickthrough.asp?url=http://robocup.org.za&id=18410&adid=P18E5

http://www.qmags.com/clickthrough.asp?url=http://www.angloamerican.co.za/media/press-releases/2013/pr16-01-2013.aspx&id=18410&adid=P18E4

http://www.qmags.com/clickthrough.asp?url=http://www.statssa.gov.za/publications/P0302/P03022013.pdf&id=18410&adid=P18E3

http://www.qmags.com/clickthrough.asp?url=www.prasa.org&id=18410&adid=P18E2

http://www.qmags.com/clickthrough.asp?url=www.robmech.co.za&id=18410&adid=P18E1






T he AlterG (T ibion) Bionic Leg W ins the IERA 2013 Award

By Raj Madhavan

The ninth annual Invention and Entrepreneurship Award in Robotics and Automation (IERA) was presented to Robert

Horst of AlterG, Inc. (formerly Tibion Corporation) for the Tibion Bionic Leg at the International Conference on Robotics and Automation in Karlsruhe, Germany (Figure 1). In 2005, the IEEE Robotics and Automation Society (RAS) and the International Federation of Robotics (IFR) agreed to jointly sponsor the Invention and Entrepreneurship Award. The purpose of this award is to highlight and honor the achievements of

the inventors with valu-able ideas and entrepre-neurs who propel those ideas into world-class products. Simultaneously, the joint disposition of the award underlines the determination of both organizations to promote stronger collaboration between robotics science and robotics industry. The award presented annually consists of a plaque and a US$2,000 honorarium.

The AlterG Bionic Leg is a wearable, battery-powered, robotic mobility-assistance device. It is a robotic trainer that is activated by the patient’s intent

to move. The device is used by physical therapists for patients with impaired mobility and is designed to strengthen stance, improve gait, and enhance active motor learning while protecting its users.

Figure 1. From left: Peter Staecker, IEEE president; Alexander Verl, IERA awards committee member; Robert Horst, AlterG, IERA Award winner; Martin Bühler, IERA awards committee member; and Raj Madhavan, IERA awards chair.


7th

Annual

October 23-25, 2013Santa Clara Convention CenterSanta Clara, CA

Connect with Thought Leaders and Peers Working to Accelerate the Commercial Advancement of RoboticsThe Most Important Robotics Conference of the Year, Located in Silicon Valley, the Heart of Technology and Innovation

Business Development & Investment

New Markets & Applications

Robotics Design & Development

Enabling Technologies & Emerging Solutions

RoboBusiness 2013 Sponsors

Sponsored by

Platinum SponsorsFounding Sponsor

Silver Sponsors

Media Sponsors

robotics business review

SAVE AN EXTRA 10% ON REGISTRATION!USE PRIORITY CODE RASMAG. robobusiness.com

®


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.robobusiness.com&id=18410&adid=P19A1






“The Awards Committee had a diffi-cult choice to make, with three excellent finalists. In the end, we picked the AlterG Bionic Leg for the following rea-son as observed in the citation for the award: a breakthrough product for rehabilitation of stroke patients at an affordable price and offering a compel-ling story of an entrepreneurial journey with typical ups and downs, culminat-ing in a successful business,” com-mented Raj Madhavan, vice president of the IEEE-RAS Industrial Activities Board and Chairman of the IERA Awards Committee. The other IERA Aw a rd s C o m m i t t e e m e m b e r s were Erwin Prassler (Hochschule Bonn-Rhein-Sieg), Martin Buehler (VECNA Technologies), Alexander

Verl (Fraunhofer IPA), and Nicola Tomatis (Bluebotics SA).

“I was very happy to accept this award for the AlterG Bionic Leg because it acknowledges the growing importance of robotics for rehabilitation. It is gratify-ing to receive such a prestigious award after our team has worked so hard to perfect the bionic leg and introduce it into physical therapy. I am looking for-ward to working with the expanded AlterG team to accelerate the availability of bionic leg therapy and develop future innovations in rehabilitation robotics,” stated Robert Horst of AlterG.

The awards committee cited the other two finalists as follows:● The Thymio II Educational Robot by

Stéphane Magnenat, ETH Zürich and

Fanny Riedo, EPFL, Switzerland: “An inspiring product motivating young boys and girls to enter science, tech-nology, engineering, and math disciplines.”

● Intelligent Grit-blasting Robots for The Surface Preparation Industry by Professor Dikai Liu, University of Technology, Australia: “A good exam-ple of bringing robotics and automa-tion technologies to tackle dull, dirty, and dangerous tasks, and to improve infrastructure maintenance.” A list of previous winners is avail-

able at http://www.ieee-ras.org/industry-government/ifr-forum/69-awards-recognition/society-awards/63-ieee-ifr-invention-and-entrepreneurship-award.

Stepping into the Future of BionicsBy Robert W. Horst

The application of robotics tech-nology to physical rehabilitation is in need of a revolution. Many years ago, my thoughts along

these lines began during my rehabilita-tion from knee surgery. While spending a few weeks on crutches, I thought there must be a way to apply robotics technol-ogy to mobility and rehabilitation needs. Finally, many years later, when I started working on a product idea, I still had no idea about most of the technology and business challenges that would need to be solved before bringing such a prod-uct to the market.

After the initial research, in 2002, I found a partner and we formed a com-pany called Tibion (for Tibia + Bionics). The goal was to develop the product then known as the PowerKnee, and today called the AlterG Bionic Leg. We slowly built a small but talented team to cover most of the aspects of hardware, software, mechanical engineering, and biomechan-ics needed to complete the project.

The product was to be a battery-powered wearable orthotic brace to help



M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________________________

_________________________

_________________________

_____________________________

http://www.qmags.com/clickthrough.asp?url=http://www.ieee-ras.org/industry-government/ifr-forum/69-awards-recognition/society-awards/63-ieee-ifr-invention-and-entrepreneurship-award&id=18410&adid=P20E1

http://www.qmags.com/clickthrough.asp?url=http://butterflyhaptics.com&id=18410&adid=P20A1






augment the quadriceps muscles to help those with impaired strength or control of one leg. Early analysis showed that the biggest technological problem would likely be the actuator, simply because the forces to help lift a patient are quite sub-stantial, requiring over 80 nm of torque at the knee joint from an actuator that should be as light as possible, ideally weighting less than 1 kg. The high torque is needed to assist in such every-day activities as sit–stand transfers and stair ascent, but the actuator must also allow normal walking while permitting the leg to swing freely and quickly. These conflicting requirements for high torque and high speed could not be met using the available technology, and we worked on nine different approaches, including various electrostatic motors and several types of continuously variable transmis-sions, before finding a breakthrough solution. The final product design uses a pair of inexpensive brushless dc motors in a patented system architecture to effectively build a high-speed, high-torque, battery-powered, electroni-cally controlled, continuously variable linear actuator.

However, developing the actuator was only part of the challenge. Once we had a working actuator design, other difficult engineering challenges still needed to be met, such as coupling the device securely to the leg, making a single device to work on the left or right legs of small and large patients, and developing noninvasive sensors and control algorithms to deter-mine and respond to the patient’s intended movements (Figure 2).

All during the engineering develop-ment phase, we were still not sure how patients would respond. The first clear indication of benefits came during our first tests with brain injury patients in 2008. The patients were at first hesitant to try the device but were soon excited to move around under their own control while being assisted. Then, as now, the first reactions to using the device are often quite emotional. Sometimes patients who have been mostly confined to a wheelchair for years are able to move around again for the first time since their stroke or injury. Tears are shared among patients, caregivers, and therapists alike

when they realize they now have new hopes for recovery.

After the first successful tests with patients, the work of commercializing the system still lay ahead. We developed the quality systems needed to meet the

Food and Drug Administration (FDA) requirements for manufacturing a med-ical device, ramped up a production facility, and developed our marketing and sales strategies to address what we knew to be a huge potential market. In the United States alone, there are nearly 800,000 stroke patients every year, and more than 400,000 of those patients require postacute care for rehabilitation of mobility deficits.

As the product matured and we headed for a commercial launch, we completed the first clinical studies that quantitatively demonstrated significant improvements in poststroke and partial spinal cord injury patients following tra-ditional therapy augmented by use of the bionic leg. The improvements shown were with patients in a chronic state (more than one year after the injury),

Figure 2. The AlterG Bionic Leg.

(continued on p. 102)

Free downloads and evaluation packages

www.reflexxes.com

Reflexes for Robots

Instantaneous & deterministic reactions to unforeseen sensor signals

Smooth motion generation within one millisecond

Multi-platform support (QNX, VxWorks, Linux, Windows, Mac OS X, and more)

Free for academic use

ReflexxesMotionLibraries

Robust, reliable, broadly tested,

and utilized by renowned robot

manufacturers.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.reflexxes.com&id=18410&adid=P21A1





TC SPOTLIGHT

22

Agricultural RoboticsBy John Billingsley, University of Southern Queensland;

Denny Oetomo University of Melbourne; and John Reid, John Deere


Service Robotics (The Rise and Bloom of Service Robots)

By Hadi Moradi, Kaz Kawamura, Erwin Prassler, Giovanni Muscato, Paolo Fiorini, Tomomasa Sato, and Radu Rusu

e may think that robots were invented to serve

humans. Consequently, what is the disparity

between the terms service robots and service robotics? Although this is a valid point, to distinguish from the initial wide usage of robots in manufacturing, the term service robotics was invented to show robotics technologies and applications in nonmanufacturing areas. The term service robots was intended to highlight emerging mar-kets for the new types of robots. This was the motivation behind initiating the Service Robots Technical Commit-tee (TC) within the IEEE Robotics and Automation Society (RAS) in 1995. During the same period, the term intel-ligent robots appeared in the literature (see [1]) to represent the new trend away from the narrow focus of the robotics community on the control of the robotic manipulators.

The Definition of Service RoboticsAlthough there is no internationally accepted definition for the term service robot, it can be defined as any robot that is not used in the manufacturing setup. The International Federation of Robotics defines a service robot as a semi- or fully autonomously operated robot that performs services useful to humans and not used for manufactur-ing [2]. A recent publication on a road-map for U.S. robotics [3] defines ser-

vice robots as those robotic systems that assist people in their daily lives at work, in their houses, for leisure, and as a part of assistance to the handicapped and the elderly.

In other words, any robot used in the medical, health care, military, domestic, and educational industries is considered a service robot. A successful example is the vacuum cleaner robots, which started with the introduction of iRobot’s Roomba in 2002, which was followed by several other robotic vacuum clean-ers that have been sold worldwide.

The History of the Service Robots TCThe founders of the TC include Kaz Kawamura as the TC chair, representing North America; Erwin Prassler, repre-senting Europe; and Tomomasa Sato, representing Asia. Table 1 shows the list of cochairs and their service terms.

To better encourage and inform the researchers interested in service robot-ics, several activities were conducted, and the TC Web site was launched in the first few years. For instance, the very first workshop, “Intelligent Planning and Control Systems for Service Robots,” was held at the 1996 IEEE International Conference on Robotics and Automation (ICRA) in Minneapolis, Minnesota. In 2000, the ninth volume of the Journal of Autonomous Robots was guest edited by Fiorini, Kawamura, and Prassler and was designated to cleaning and housekeeping robots [4]. In the same year, Prassler and Fiorini conducted a tutorial “Cleaning and Housekeeping

Robots” at ICRA in San Francisco. These activities were followed by a special session on robot assistants at ICRA 2001, the first international con-test of cleaning robots at the IEEE/Robotics Society of Japan (RSJ) International Conference of Intelligent Robots and Systems (IROS) in 2002, and a special session on service robots at IROS in 2004.

As Kawamura recalls, it was an excit-ing time to lead the community into this new field, and this excitement and hard work was recognized by RAS when it awarded the TC the prestigious Most Active Technical Committee Award in 2003.

In addition to the general activities conducted by the TC, such as conduct-ing workshops and supporting confer-ences, one of the activities that the Ser-vice Robots TC highly supported was the initiation of the Industrial Activities Board (IAB) and its Innovation and Entrepreneurship in Robotics and Automation (IERA) forum and award. Since the IAB’s goals were very much related to the goals of service robots,


Table 1. The list of Service Robots TC cochairs.

Kaz Kawamura 1995–2006

Erwin Prassler 1995–2006

Tomomasa Sato 1995–2006

Paolo Fiorini 1997–2006

Hadi Moradi 2006–2013

Giovanni Muscato 2006–2013

Radu Rusu 2009–2013

W


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







the cochairs of the Service Robots TC were very much involved in its activities. For instance, the IERA, which was conducted through a pro-ductive collaboration between RAS’s IAB and the Interna-tional Federation of Robotics, began in 2005 to encourage commercialization of robotics ideas. Furthermore, in 2006, the standardization commit-tee was founded in IAB, and headed by Prassler, to facili-tate the acceptance of service robots in the commercial market [5]. This activity is in collaboration with the ISO’s working groups on service robots standardization, i.e., WG7, WG8, and WG9. The roadmap activity for intelligent service robots was another approach supported by the Service Robots TC to improve research and development toward widespread use of service robots [6], [7].

Although the TC’s Web site was created in the 1990s, in 2006, it obtained a new face, with more information on research labs and commercial sectors in the field of service robots [8]. In 2008, the TC started its news group, and the TC continued to support workshops in emerging areas such as space robot-ics. The last activity of the TC was the support of the Robotics Contest for Innovative Industrial and Municipal Application at ICRA 2013, organized by the Robotics Society of Iran and the Iranian Society of Mechatronics (RSI/ISM), in which all the submitted designs were for nonindustrial applications.

Retirement TimeAfter 15 years of effort to encourage the community toward this new area, the TC was fairly mature. Related committees, such as the Autonomous Ground Vehicles and Intelligent Transportation Systems TC, were established. Figure 1 shows the number of TCs that have been created that are related to ser-vice robotics, which has increased

significantly since the start of the Ser-vice Robots TC.

In addition, the professional and personal service robots statistics [9] show steady growth in the use of ser-

vice robots. The total number of profes-sional service robots sold in 2011 was 16,408 units compared with 15,027 units in 2010. It is worth mentioning that the total number of professional

www.ati-ia.com/ras

Standard Features: Six Axes of Force/Torque Sensing (FxFy Fz Tx Ty Tz) High Overload Protection Interfaces forEthernet, PCI, USB, EtherNet/IP, CAN, and more Sizesfrom 17 mm - 330 mm diameter Custom sensors available

Applications: Product Testing Biomedical ResearchFinger Force Research Rehabilitation Research Robotics

Six-AxisForce/Torque Sensors

Smart BuildingsSoft Robotics

Marine RoboticsHaptics

Space RoboticsHumanoid Robotics

BioroboticsAerial Robots and Unmanned Aerial Vehicles

Autonomous Ground Vehicles and ITSSafety, Security, and Rescue Robotics

Teleoperated Networked RobotsMicro/Nano Robotics

Human–Robot InteractionAgricultural Robots

Service Robots

1995

1997

1999

2001

2002

2004

2005

2006

2007

2008

2012

Figure 1. The initiation and termination of the TCs related to service robots.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.ati-ia.com/ras&id=18410&adid=P23A1






robots sold in 2010 and 2011 constitutes 29% of the total units sold between 1998 and 2011, i.e., 110,000 units, representing the sig-nificant growth of service robot usage since 1995.

Figure 2 shows the statis-tics of professional service robots sold in 2010 and 2011. In Figure 2, the robots sold in construction, mobile platforms, cleaning, inspec-tion, underwater, and search and rescue are not shown because of the small number of units sold. One of the very small mar-kets that needs further attention is handicap-assistant robots, with 156 units sold in 2011 and 46 units in 2010. Because these robots are very high-tech, they need further research to pre-pare them for the market.

Finally, it is interesting to mention that the number of robots sold in the personal and domestic market grew 15% in 2011 compared with 2010, i.e., from ~2,170,000 to 2,500,000.

The FutureAlthough the Service Robots TC is retir-ing, the TC cochairs, past and present, believe that the following areas will grow

and will need more attention from RAS: 1) educational robotics, 2) domestic ser-vice robots, and 3) intelligent compan-ion robots such as intelligent toys.

For more information, contact the corresponding author, H. Adi Moradi,

School of Electrical and Computer Engineering, Uni-versity of Tehran, at [email protected].

References[1] K. Kawamura, S. Bagchi, M.Iskarous, and M. Bishay, “Intelligent robotic systems in service of the disa-bled,” IEEE Trans. Rehab. Eng., vol. 3,no. 1, pp. 14–21, 1995.[2] Internat iona l Federat ion of Robotics. Service robots: Provisional definition of service robots. [Online]. Av a i l a b l e : h t t p : // w w w. i f r . o r g /service-robots/[3] H. I. Christensen. (2013, Mar.). Aroadmap for US robot ics: From

Internet to robotics. [Online]. Available: http://www.robotics-vo.us/[4] P. Fiorini, K. Kawamura, and E. Prassler,“Cleaning and household robots,” J. Auton. Robot.,vol. 9, no. 3, pp. 207–209, Dec. 2000.[5] S. Lee, E. Prassler, R. Tuokko, and H. Moradi,“The standardization and roadmapping initiatives by the industrial activity board” IEEE Robot. Automat. Mag., vol. 12, no. 4, p. 98, Dec. 2005.[6] S. Lee and H. Moradi, “RAS IAB’s roadmap work shop on i nte l l igent ser v ice robots ,”IEEE Robot. Automat. Mag., vol. 12, no. 1, p. 12, Mar. 2005.[7] H. Moradi, “Roadmaps for Robotics and Automation,” IEEE Robot. Automat. Mag., vol. 16,no. 3, p. 98, Sept. 2009.[8] Service Robots Technical Committee. Robotics and Automation Society. [Online]. Available:http://www.service-robots.org/[9] International Federation of Robotics. (2012). Service robot statistics. [Online]. Available: http://www.ifr.org/servicerobots/statistics/

The use of soft and deformable materials in robotic systems has increasingly gained inter-est in recent years. Because

of its potential to deal with uncer-tain and unstructured task environ-ments, soft-body robotic systems are expected to be able to accomplish tasks such as grasping and manipu-lation of unknown objects, locomo-tion in rough terrains, and perform-

ing flexible interactions between robots and living cells or human bodies. In addition, soft robotics pushes the boundary of visionary research topics such as growing, self-repair ing, and sel f-replicating robots. Examples of the recent major achievements in soft robotics are shown in Figure 1.

As compared with other robotic systems, soft robots are characterized by a number of unique aspects: 1) elastic and deformable bodies, 2) a large number of degrees of freedom,

3) use of unconventional materials to compose the body, and 4) involvement of intrinsic passive mechanical dynamics. Related to this, creating soft robots also increases our understand-ing of a modern view of intelligence, the so-called embodied intelligence or morphological computation, which gives more importance to the role of the body and its physical interaction with the environment.

The IEEE Robotics and Automation Society Technical Committee (TC) on Soft Robotics was established in October

Soft RoboticsBy Surya G. Nurzaman, Fumiya Iida,

Cecilia Laschi, Akio Ishiguro, and Robert Wood


0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

Defense Field Logistics Medical

Figure 2. The number of professional service robots sold in 2011 (blue) and 2010 (red).

Figure 3. The Service Robot TC retirement ceremony at ICRA 2013: TC Chairs Erwin Prassler and Paolo Fiorini with RAS President David Orin, center.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


____________________

_____________

___________________

_______

____

http://www.qmags.com/clickthrough.asp?url=http://www.ifr.org/service-robots/statistics/&id=18410&adid=P24E1

http://www.qmags.com/clickthrough.asp?url=http://www.service-robots.org/&id=18410&adid=P24E4

http://www.qmags.com/clickthrough.asp?url=http://www.robotics-vo.us/&id=18410&adid=P24E3

http://www.qmags.com/clickthrough.asp?url=http://www.ifr.org/service-robots/&id=18410&adid=P24E2







2012 to bring together scientists and engineers with different backgrounds and disciplines, providing the opportu-nity to share the latest publications, tech-nologies, experiences, and other relevant information among those with a similar interests in soft-body robotic systems.

The Organizational Structure and Priority AreasThe main organizational structure of the TC consists of the four cochairs and several founding members. The

current cochairs of the TC are Fumiya Iida, Cecilia Laschi, Akio Ishiguro, and Robert Wood. The founding members are Dario Floreano, Rolf Pfeifer, Carmel Majidi, Kyujin Cho, Hod Lipson, Daniela Rus, Barry A. Trimmer, and Maarja Kruusmaa.

As the first crucial effort to organize the TC, it was decided that the priority areas for the TC would include 1) sci-entific problems related to soft-bodied robots, 2) soft materials for robots, 3) soft actuators and sensors, 4) modeling

and simulation techniques of soft bod-ies, 5) fabrication and control of soft bodies, 6) interdisciplinary interaction with biological/medical sciences, mate-rial sciences, chemistry, and other related disciplines, and 7) soft robotics applications.

Activities Organized by or Related to the TCToward the establishment of the TC, a wide range of activities has been initi-ated in the last few years:● Organized sessions on soft robotics

and on smart materials and actua-tors for soft robotics in the IEEE International Conference on Bio-medical Robotics (BioRob 2012), 24–27 June 2012

● E i d ge n ö s s i s c h e Te c h n i s c h e Hochschule (ETH) Summer School on Soft Robotics (Figure 2), Zürich, Switzerland, 18–22 June 2012

● Special Issue on Soft Robotics in Journal of Advanced Robotics 26(7),2012

● Special Session on Soft Robotics at the 2011 European Future Technolo-gies Conference (FET11), Budapest, 4–6 May 2011

● Swiss–Japan Joint Seminar on Soft Robotics: Morphology, Materials, and Functionalities, University of Tokyo, 20–23 June 2010.Since its official formation, the

TC has been organizing the activities in a more effective way and, in 2013 alone, there are several major events:● International Workshop on Soft

Robotics and Morphological Computation, Monte Verità, Ascona, Switzerland, 14–19 July 2013

● Organized sessions on soft technolo-gies for wearable robots at the IEEE International Conference on Intelligent Robots and Systems (IROS), 3–7 November 2013.Because of the nature of soft robotics

as an interdisciplinary field, the work-shop at Monte Verità, for example, will be attended by more than 100 partici-pants from all over the world with 17 invited speakers from various scien-tific disciplines. The speakers include

(a) (b)

(c) (d)

Figure 1. (a) A soft gripper based on jamming granular material, which is able to pick up unfamiliar objects with a widely varying shape and surface, (b) an octopus-inspired robot whose arms are modeled based on the characteristic muscle of the octopus, (c) a caterpillar-inspired soft bodied rolling robot, and (d) a foldable robotic system based on the concept of origami design. [Images courtesy of Hod Lipson (Cornell University), Cecilia Laschi (SSSA; image: London Science Museum/Jennie Hills), Barry A. Trimmer (Tufts University), and Jamie Paik (EPFL).]

Figure 2. The participants of ETH Summer School on Soft Robotics, June 2012.

(continued on p. 95)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Autonomous Underwater Biorobots

Tareq Assaf, Cesare Stefanini, and Paolo Dario

This article describes a new design for wireless power transfer in autonomous underwater robots. The aim is to propose a solution for battery charging by taking into account the morphological and dimensional constraints of robots requiring small and low-weight internal modules. An innovative design is presented for inductive power transfer suitable for a wide range of applications. The system is

conceptually equivalent to a transformer in which the core can be separated into two parts during operation, one for each coil. Inductive power transfer is selected to have a system to easily and reliably charge different kinds of underwater robots. The secondary coil and its magnetic core are designed to be placed inside a bioinspired robot; the weight, dimensions, and power output for battery charging are optimized. The shape of the secondary magnetic core section is hollow to house the control electronics and sensors. The primary coil is the power inductor, which is placed in a docking unit outside the robot. Experimental results are also reported.

Project LAMPETRAThe framework for the development of the device described in this article is the project Life-Like Artifacts for Motor-Postural Experiments and Development of New Control Technologies Inspired by Rapid Animal locomotion (LAMPETRA), a three-year (2008–2010) collaborative

A Wireless System for Power Transfer PHOTO COURTESY OF TAREQ ASSAF,

CESARE STEFANINI, AND PAOLO DARIO

Date of publication: 1 August 2013Digital Object Identifier 10.1109/MRA.2012.2201577


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







research project funded under the Seventh EU framework program, theme ICT-2007.8.3-FET proactive 3, “Bio-ICT Convergence,” aimed at consolidating multidisciplinary Bio-ICT research carried out by a recently established scientific community. It involves four European countries: Italy, Swe-den, France, and Switzerland. The objective of the project LAMPETRA is to develop lamprey and salamander bioin-spired artifacts as tools for neuroscientific studies on goal-directed locomotion. The partners have already developed early prototypes: a lamprey-like artifact by Scuola Superiore Sant’Anna—Pisa (SSSA) [1], [2] and a salamander-like artifact [3], [4] by the Biologically Inspired Robotics Group-Lausanne—Switzerland (EPFL). In the lamprey prototype (SSSA), both power supply and con-trol were wired. This article focuses on the adoption of on-board batteries to enhance free locomotion. It also addresses the on-board integration of control electronics and sensors. In this article, a wireless solution for battery charging is described, as a number of constraints are provided by the artifact. The robot is covered by a smooth thin waterproof skin layer, which is supposed to be deli-cately touched during a charging process carried out through a non-contact power transfer. In addition, issues such as body flexibility and space for internal electronics mean that there has to be a bulk miniatur-ization (axial and transversal) of the on-board power receiving module. The device introduced in this article is a transformer-based system that can transfer electric energy from a burrow-like docking station to the robot, for feeding through battery charging.

State of the Art

Docking and Power Transfer System Despite the wide range of theories, methodologies, and devices for wireless power transfer described in the literature, we focus on those solutions adopted in the field of oceanic exploration for autonomous underwater vehicles (AUVs). Shape, dimensions, and energy requests change case by case, and the working principles of the charging systems can be grouped into three classes: direct electric contact, electrome-chanic generators, and inductive transfer [5], [6].

In the first type, energy is usually transferred to electric contacts powering an on-board isolated transformer. The contacts can be damaged by oxidation and mechanical stress or by other interactions with the environment, and they are also a critical in terms of waterproofing. The second system

(electromechanic generators) can be applied to vehicles that use propellers and are less suitable for bioinspired robots that perform swimming by body deformation. The electric energy is transferred from the docking station to the robot by a rotating shaft. A motor in the docking station turns the shaft that is coupled to a generator or to an alternator in the AUV. Conceptually, the third system (inductive transfer) is a transformer where the core can be split into two parts. It exploits an alternating electromagnetic field to transfer the energy from a primary coil, placed in the docking station, to

a secondary coil, in the AUV. The power is transferred across the air/water gap between the two coils. A transformer-based system was there-fore selected as the best solution for the given application.

Design

Robot Overview The prototype of the lamprey-like artifact is supposed to be autono-mous, not only in terms of behavior but also concerning energy and mobility. To meet these require-ments, the robot is equipped with 12 vertebras, four active segments, six Lipo batteries, and a large number of sensors, such as vision, vestibular, and light sensors (Figure 1). Because of the critical dimensions of the robot, it was necessary to find solu-tions to different integration issues to include all the components on board.

As can be seen in Figure 2, the robot can be divided into three main parts: the head, the segments, and the tail. The tail provides a significant contribution to the propulsion. It is a passive element that transmits the locomotion wave to the water with high efficiency. The segments are actuated by novel muscle-like actuators contained within a number of vertebras, which also host central pattern generator electron-ics and batteries. The vision and vestibular systems, the high-level control electronics, and the recharging device are all placed in the head. The head section represents the central computational unit. Placing all the above-mentioned compo-nents in the head was a big challenge considering the con-straints imposed by the project.

Figure 1. The head segment. The vision, the central board, and the secondary winding for the wireless charge system are placed inside this module.

Head (11 cm) Segments (55 cm) Tail (16 cm)

Figure 2. The entire robot without the skin cover. The active vertebras and the batteries modules are visible in the segments section.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Device Overview A transformer-like system that can transfer energy from a dock-ing station to a robot is described here. The system is a kind of transformer with a split core: 1) the primary coil (the source) and 2) the secondary coil (the receiver). Constraints due to the bioinspired approach during the artifact development are on the basis of the design of the device.

Low weight and low encumbrance are the most important features to be taken into account to host the secondary coil within the robot. The main criterion in the design is the max-imization of the ratios, transferred power versus weight, and size. Efficiency is of secondary importance for our purposes and is just considered for heating issues. The novelty of the

system is the hollow shape of the secondary magnetic core section in which the electronics can be hosted.

The inductive power transfer was chosen after considering the advantages and drawbacks of the systems in the literature.

Model The secondary coil was modeled first to fit the constraints imposed by the size and shape of the robot, which is eel shaped and composed of many segments. The diameter is around 50 mm, and the length of the robot depends on the number of segments (Figure 3). The charging system is supposed to be placed in the head segment with a length of 70 mm.

For the secondary coil magnetic core section, two dif-ferent designs were considered to reduce the bulk and the weight and to maximize the transferred power. Figure 4presents the two possible approaches. The two repre-sented metal cross sections have the same cross area, but Case 1 has a uniform metal core, whereas Case 2 has a hollow shape. Case 1 is sometimes used for underwater vehicles [8], [9]. However, this solution does not fit our case because of the constraints related to bioinspiration. For example, with the increase in the air gap, the magnetic inductance would be reduced too much, and the elec-tronic components would have to be placed around this structure, which would result in a difficult design.

On the other hand, Case 2 has more free room available for the electronics board, and the magnetic core is attached to the head shell, thus minimizing the gap with the external inductor.

The resulting longitudinal section envisaged for the device is shown in Figure 5.

The secondary magnetic core section is separated from the primary section by a coupling interface, which is the air/water gap. The primary metal core section, consisting of six ele-ments, surrounds the secondary.

Design constraints are the overall dimensions and mass, turns, magnetic flux, and desired power. After a parametric study reported in the following, we chose the optimum working point for our application in terms of maximum delivered power to weight ratio. The primary coil was dimensioned after the secondary, but in this case, dimen-sions were not a constraint because coupling geometry (interface with the secondary magnetic core section) and electromagnetic flux generation were the only two require-ments. Again, we chose the best compromise between power, dimensions, and number of turns.

Equations and Dimensioning Figure 6 shows dimensioning parameters of the cross section of the secondary core shape.

To design the secondary coil, we started with the sec-ondary coil core design. Equations (1) and (2) describe the core geometry in relation to the length and diameter, respectively. The magnetic flux flows from the primary coil to the secondary coil in the magnetic core, crossing the lit-tle air gap between the two (Figure 5). In the cross section

Head

Artifact

Tail

SecondaryWinding Control

Electronics

Segments

Figure 3. LAMPETRA artifact. The first segment is the head-like section. This hosts the charging system and the control electronics. The total length of the robot is around 80 cm depending on the number of segments and on the tail length. Every segment implements magnetic muscle-like actuators [7] and local control electronics.

Usable Room

Case II

Robot Skin

Case I

Figure 4. Comparison between the designs of the secondary magnetic core section. The room available and the coupling in Case 2 are better than in Case 1.

Primary Core Section

Coupling Interface

Secondary Area forCoil Windings

Secondary Core Section

Magnetic Flux

Figure 5. Longitudinal section of the device shows the architecture of the transformer. The secondary magnetic core section is coupled with the primary, which is divided into six sections mounted around the secondary to maximize the coupling area.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







(Figure 6), h and ( )c r , where /r b 2= , are dependent vari-ables for the wire thickness and diameter of the device, respectively. These dependencies enable the magnetic flux to be maintained constant at each point of the magnetic core. We approximated these values to a constant size in this prototype.

( ) ,C r rbd b

22

= + (1)

.h Dbd b2

= + (2)

The maximum number of turns ( )Nt was calculated using (3) by varying the inner diameter ( )d and considering the fol-lowing values: external diameter ( )D 50 mm, diameter of iron strips ( )b 1 mm ( / ),r b 2= length ( )L of the device 60 mm, and copper wire diameter ( )d f 0.5 mm:

(( ) /( )) .Nd

L a bd b b d2 2 2 222

2

t $=- + + (3)

The voltage ( )VEMF sec; and inner resistance ( )R are calcu-lated using (4) to estimate the output power ( )Powu estimated in (6) and the current transferred to the secondary coil:

,R N l S Nd

D d b1 2 22t t

ft

f$ $ $t t= = + + (4)

| ( ),V N f B bd b2EMF2 2

sec maxt $ $r= + (5)| ,Pow V

R2 41EMF sec

u $= (6)

where t is the resistivity, f is the frequency, Bmax is the maximum magnetic field for the given material, and lt is the turn length. Relevant characteristics, plotted in Fig-ure 7, were evaluated to choose the dimension of the inner diameter. We chose the maximum power/mass ratio as working point.

The primary coil consists of a number of subcores (six) that surround the secondary coil. The primary coil was designed to ensure an adequate magnetic flux in the second-ary coil, by exploiting (1)–(6) and the following relations:

( ) .Ni D 0{= (7)

The general law governing the relation between the number of turns ( ),ND the current ( ),i and the magnetic flux are represented by (7). The reluctance ( )0 calcu-lated by using (8) is an intrinsic value of the magnetic core, depending on its shape. Air reluctance is calculated by approximating the air gap size:

.2tot air/water prsec0 0 0 0= + + (8)

ND is calculated by using (9) considering the free space around the magnetic core where W is the width of wind-ing, L is the length, and Aw are “wing” dimensions (refer to a in Figure 6):

( ) .Nd

W L A22Df

w=

- (9)

Resistance was estimated in

.Rl

N l42f

D t

r

t= (10)

D

L

c(r)b

d

ahf

L

c(r)bb

d

ahf

Figure 6. Secondary cylindrical magnetic core unit cross section. How the laminations were shaped for the secondary magnetic core section is shown. The external diameter ( ),D the column diameter ( ),b the wings for coupling with the primary core ( ),aand length of the device ( )L are given data. The wing diameter ( ( ))h d and the winding area ( ( ))c r are dependent variables ( / ) .r b 2= The inner diameter ( )d is the independent parameter. The magnetic flux is .f

Pow_u

Veff_load

leff_load

N_t/100

Mass [hg]

Pow_s

d00

5

10

0.01 0.02 0.03 0.04 0.05 0.06

Figure 7. Output power, voltage, current, number of turns, mass, and power/mass ratio are plotted as a function of the inner diameter d,considering the external diameter, the axial length, and the diameter of the wire (0.5 mm) as given. Arrow: the working point chosen, whereas the related obtained values can be found in Table 1.

Table 1. Working values obtained from the model.

Parameters Values (IU)

Nt , number of turns 560

Veff load in voltage 7.63 (V)

R, inner resistor 7.07 (X)

Ieff load, current 1.08 (A)

Powu , output power* 8.23 (W)

M, mass* 0.22 (kg)

Pows , power/mass ratio* 36 (W/kg)

Marked (*) parameters are not dependent on the wire diameter (df). Other parameters are strictly related to df. The wire diameter was chosen considering the current and voltage desired for the application.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Materials and Methods

Secondary Module As already mentioned, the secondary coil dimensions are approximately 50 mm in diameter and 60 mm !2 mm in length. Off-the-shelf materials were chosen as building elements.

Materials: ● iron wire (1.2 mm diameter) ● Delrin ● copper wire (0.5 mm diameter) ● epoxy material.

The iron wire was shaped into curved laminations (Figures 8 and 9) in accordance with the transformer theory. In fact, using a low-frequency magnetic field (50 Hz), hysteresis losses and eddy currents can influence the device by generat-ing side effects in the magnetic core. To reduce these effects, which may decrease the efficiency of the device, the magnetic core needs to be built with lamination-like elements.

We used a Delrin core to assemble the metal laminations in the cylindrical-shaped ending (Figure 10). Epoxy material was used to create a rigid structure and to provide electronic insu-lation. The magnetic core received further electric insulation by means of insulating lacquer. The module was then wound in copper wire. More than 500 windings were utilized. Weight, resistance, and dimensions were checked after fabrication.

Primary Module The primary unit was developed by following similar design criteria to those adopted for the secondary side, particularly with regard to the maximum magnetic induction available in the material and target voltage. The module is composed of six radial coils (Figure 11), generating a magnetic flux, and poles are located with a small radial gap at the ends of the sec-ondary module. Epoxy material was used to link iron pieces in the six cores. Coils were made by using a wire with a diam-eter of 0.8 mm, which gives an impedance for the six coils in series that matches the voltage given by a commercial transformer, as detailed in the following.

After preliminary tests on overall functionality, the mod-ule was embedded in polyurethane (PU) to strengthen the structure, increase the heat dissipation, and make the system waterproof. The device will also be coated with a special lac-quer paint to obtain a smooth surface and to ensure complete isolation when underwater.

Experiments Once the final prototype was ready (Figure 12), experi-ments were performed to verify compliance with the model and to characterize the device. Particular attention was paid to the testing of the output power supplied by the secondary coil. Table 2 summarizes the main charac-teristics of the system.

The complete device is powered by an external worksta-tion and works at 48 V and 50 Hz. The most important value to investigate is the maximum output power of the second-

Figure 8. The final and intermediate shape of the iron wire.

Figure 9. Iron wire modeling. About 200 laminations were crafted for the entire device. A simple but stable setup was built to ensure constant dimensions for all the laminations.

Figure 10. The secondary coil is mounted using the laminations, and a Delrin core is used as a building frame for the metal. First, the metal is bound in a solid structure using Epoxy material and then wound in copper wire.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







ary coil. The output power is maximum when load imped-ance equals the conjugate inner one according to the maxi-mum power transfer theorem. An oscilloscope (Figure 13)was used to measure the voltage at the ends of a load: this is a series of resistors and capacitors whose resulting impedance is close to the conjugate of inner resistance and inductance.

The results of the tests are summarized in Table 3. The “Real Device” column shows the experimental data obtained with the device, and the “Model” column shows the data on the theoretical dimensions retrieved using the model. The real values are very close to the parameters obtained with the model. Finally buoyancy, as given by the Archimedes princi-ple, was estimated: the mass of the secondary device is 200 g, and its overall volume is 145 cm3; therefore, a negative trim of 75 g is present and must be negotiated by exploiting small additional adjacent empty spaces.

The efficiency, as reported on Table 3, is roughly 16% and is close to the estimated value. The low efficiency of the device is due to the design in terms of operating frequency and also the handmade lamination and windings. Efficiency

can be increased by tuning the frequency and by improving the magnetic core. On the other hand, the aim of this device is not to be highly efficient, above all to be suitable for bioin-spired purposes in water environment and to be safe. From this point of view, the device works as expected and is fully compatible with the electronics and all other robot compo-nents. Finally, the energy requirements of the robot are very low (power consumption is less than 5 W, and battery capacity on board is only 20 Wh) and therefore energy before conversion is also very limited.

DiscussionThe primary coil can theoretically work up to a standard volt-age of 220 V at 50 Hz; however, it can work at a lower voltage and was developed to operate at 48 V. By reducing the voltage: 1) the device is less dangerous in a water environment, 2) the energy required is lower, and 3) the heat dissipation decreases. The work frequency is also an important parameter. Although an increase in frequency improves the performance of the device, the design proposed here works at 50 Hz, considering the international standard frequency (50–60 Hz); the device can thus be plugged in easily everywhere. These design choices

Tek

M 10.0 ms

M Pos: 0.000s Measure

CH1 RMS9.03 V

CH1None

1

CH1Peak–Peak

27.8 V

CH1None

CH1 5.00 V CH1/4.80 V

CH2 OffFreq.

Stop

Figure 13. Output voltage amplitude is approximately 14 V.

Figure 12. Final result before coating.

Table 2. Summary of working values obtained from the model.

Data Secondary Primary Primary + PU

Mass (kg) 0.206 1.805 3.702

Resistor (X) 6.5 10 10

Electrical Isolation

Yes Yes Yes*

Dimensions (mm)

Q50 # 68 Q115 # 68 178 # 173 # 83

Air/water gap max (mm)

— 2 1.5

*Waterproof.

Figure 11. The secondary coil was used to mount the six different primary subcores at the correct distance. The secondary coil needs to be free to get in and out of the primary magnetic coil.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







are strictly related to the low efficiency of the device compared with the traditional transformers that can achieve a very high efficiency up to 95%. The air/water gap, the low working fre-quency, and the manufacturing process reduce the efficiency to 16%. This value is acceptable for our application in which external power is not an issue. Furthermore, the concept and the design are still valid, although applications would require higher performance and efficiency.

Another important feature is the hollow shape of the secondary core section. Using this design, electronics can be placed into the secondary coil magnetic core, thus limit-ing any wasted space. This design also reduces the weight, which is an important parameter in underwater applica-tions when a neutral trim is required.

The secondary coil final mass is 200 g. Considering the total length (and volume) of the artifact, this low weight enables the buoyancy of the robot to be designed, thus not having to take into account the weight of the secondary coil.

Finally, the expected self-docking feature has been observed. The secondary coil is attracted into the primary coil with a delicate force of about 0.5 N. This function is not mandatory for our application, where we have locomotion capability; however, it is still useful as additional self-guid-ance while docking.

Conclusion We have successfully developed a low-weight system for bat-tery charging by proposing a new design for wireless power transfer devices. Our design is useful in situations when there are constraints regarding: small dimensions, a cylindri-cally shaped robot, low weight, and high electrical insulation. Another advantage is that only one primary coil is needed to charge different robots equipped with individual secondary coils. Robot docking is critical and our device provides a neat solution with self-guidance. All these features highlight that our device would work well in AUV applications.

During the tests, the device did not cause any damage to the electronics when the magnetic field was applied. The device will be shortly integrated into the whole system.

Acknowledgment This work is primarily being carried out under the Euro-pean Project LAMPETRA (Life-like Artifacts for Motor-Postural Experiments and Development of new Control Technologies inspired by Rapid Animal locomotion) Proj-ect Reference: 216100 through the Seventh Frame Program research area: ICT-2007.8.3-FET “Bio-ICT Convergence.” It also comes under the European Project ANGELS (ANGuil-liform robot with ELectric Sense) Project Reference: 231845 through the Seventh Frame Program, research area: ICT-2007.8.5-FET “Embodied intelligence.”

References[1] C. Stefanini, S. Orofino, L. Manfredi, S. Mintchev, S. Marrazza, T. Assaf, L. Capantini, E. Sinibaldi, S. Grillner, P. Wallen, and P. Dario, “A novel auton-omous, bioinspired swimming robot developed by neuroscientists and bioen-gineers,” Bioinspiration Biomimetics, vol. 7, no. 2, p. 025001, 2012.[2] S. Grillner, A. Kozlov, P. Dario, C. Stefanini, A. Menciassi, A. Lansner, and J. H. Kotaleski. (2007). Modeling a vertebrate motor system: Pattern genera-tion, steering and control of body orientation. Prog. Brain Res. [Online]. 165(14), pp. 221–234, PMID: 17925249. Available: http://www.ncbi.nlm.nih.gov/pubmed/17925249[3] A. J. Ijspeert, J. Hallam, and D. Willshaw. (1999, Mar.). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology.Adapt. Behav. [Online]. 7(2), pp. 151–172. Available: http://adb.sagepub.com/cgi/content/abstract/7/2/151[4] A. Crespi, A. Badertscher, A. Guignard, and A. J. Ijspeert, “AmphiBot I: An amphibious snake-l ike robot,” Robot. Auton. Syst ., vol. 50, no. 4,pp. 163–175, 2005.[5] R. Stokey, M. Purcell, N. Forrester, T. Austin, R. Goldsborough, B. Allen, and C. von Alt. (1997). A docking system for REMUS, an autonomous under-water vehicle. presented at OCEANS MTS/IEEE Conf. Proc. 2, pp. 1132–1136.[Online]. Available: 10.1109/OCEANS.1997.624151[6] R. Coulson, J. Lambiotte, G. Grenon, T. Pantelakis, J. Curran, and A. An. (2004). Development of a modular docking sub-system for 12" class autonomous underwater vehicles. presented at OCEANS MTTS/IEEE TECHNO-OCEAN. 3, pp. 1745–1749. [Online]. Available: 10.1109/OCEANS.2004.1406388[7] C. Stefanini, S. Mintchev, and P. Dario, “Permanent magnet actuator for adaptive actuation,” Patent WO 0 15 997, Feb. 11, 2010.[8] B. Heeres, D. Novotny, D. Divan, and R. Lorenz. (1994). Contactless under-water power delivery. presented at IEEE Power Electronics Specialists Conf. PESC Rec. 25th Annu. 1, pp. 418–423. [Online]. Available: 10.1109/PESC.1994.349700[9] A. Bradley, M. Feezor, H. Singh, and F. Y. Sorrell. (2001). Power systems for autonomous underwater vehicles. IEEE J. Ocean. Eng. [Online]. 26(4), pp. 526–538. Available: 10.1109/48.972089

Tareq Assaf, Scuola Superiore Sant’Anna, Pontedera 56025, Italy. E-mail: [email protected].

Cesare Stefanini, Scuola Superiore Sant’Anna, Pontedera 56025, Italy. E-mail: [email protected].

Paolo Dario, Scuola Superiore Sant’Anna, Pontedera 56025, Italy. E-mail: [email protected].

Table 3. Test results.

Data Model Real Device

Output power (W) 12.2 12.5

Efficiency (%) 18 +16

Current (A) 1.3 1.4

Voltage (V) 9 9.5

Resistor (X) 6.4 6.5

Capacitor (nF) — 67

Phase correction (°) — +151

Dimensions (mm) Q52 # 68 Q52 # 68

Metal core diameter (mm) Q1.3 Q1.3

Turns 530 +530


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_____________

____________

_________

_____________

http://www.qmags.com/clickthrough.asp?url=http://adb.sagepub.com/cgi/content/abstract/7/2/151&id=18410&adid=P32E2

http://www.qmags.com/clickthrough.asp?url=http://www.ncbi.nlm.nih.gov/pubmed/17925249&id=18410&adid=P32E1









33SEPTEMBER 2013 IEEE ROBOTICS & AUTOMATION MAGAZINE1070-9932/13/$31.00©2013IEEE

Carp is a highly invasive bottom-feeding fish that pollutes and dominates lakes by releasing harmful nutrients. Recently, biologists started studying the behavior of carp by tagging the fish with radio emitters. The biologists search for

and localize the radio-tagged fish manually using a global positioning system (GPS) and a directional antenna. We are developing a novel robotic sensor system in which human

effort is replaced by autonomous robots capable of finding and tracking tagged carp.

In this article, we report the current state of our system. We present a new coverage algorithm for finding tagged fish and active localization algorithms for precisely localizing them. In addition to theoretical analysis and simulation results, we report results from field experiments.

Robotic Sensor SystemInvasive fish, such as the common carp, pose a major threat to the ecological integrity of freshwater ecosystems around the world. Presently, these fish are controlled with


By Pratap Tokekar, Elliot Branson, Joshua Vander Hook, and Volkan Isler

Date of publication: 11 September 2013

Autonomous Robots for Monitoring Invasive Fish

IMA

GE

LIC

EN

SE

D B

Y IN

GR

AM

PU

BLI

SH

ING


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







nonspecific toxins, which are expensive, ecologically dam-aging, and impractical in large rivers and lakes. Recent studies in small lakes have established that the common carp aggregate densely at certain times and in specific regions within the lakes [1]. Their population can be con-

trolled by targeting these aggregations using net-ting. To predict the loca-tions of aggregations within a lake, biologists surgically implant radio tags on some fish and use radio antennas to track them periodically (Figure 1). Manually locating tagged fish in

large turbid bodies of water is difficult and labor inten-sive. Our goal is to build a robotic system to automate this tedious manual task.

In our previous work [2], we demonstrated the feasi-bility of finding radio-tagged carp using a custom-made catamaran-style autonomous robot. Our current system features a new boat (Figure 1) and improved system architecture.

We perform the task of locating tagged fish in two phases: search and localization. The goal in the search phase is to find a location from which the tag can be sensed. In the localiza-tion phase, the goal is to accurately estimate the location of the fish. We have developed new search and active localiza-tion algorithms suitable for finding tagged carp under the assumption that they loiter in their home ranges for long peri-ods of time [3].

Related WorkRecent years have witnessed the development of many environmental monitoring systems that use aquatic robots. Applications include tracking dynamic phyto-plankton [5] and collecting biological and environmental data from stationary sensors [6], [7]. A recent survey by Dunbabin and Marques [8] provides an excellent over-view of such systems.

The search problem previously discussed is closely related to the robotic coverage problem, which has received significant attention [9]. We introduce a new version of the problem in which specific regions of the lake must be covered as opposed to the entire lake. We present a constant-factor polynomial-time approxima-tion algorithm for this problem (see [10] for more infor-mation on approximation algorithms).

In the localization phase, we perform active bearing-only target localization. In one of the earlier works on this problem, Hammel et al. [11] used the determinant of the Fisher information matrix (FIM) as the objective function to be maximized and numerically computed an optimal open-loop trajectory for a robot in the case in which mea-surements are obtained continuously. The resulting trajec-tory follows a spiral shape, but is an open-loop trajectory that does not depend on the actual measurements the robot obtains. Frew [12] presented a feedback strategy for tracking targets with bearing measurements obtained using monocular vision. This strategy is based on a state-exploration tree, and a trajectory is obtained using a breadth-first search for the minimum uncertainty. Recently, Zhou and Roumeliotis [13] considered the active localization problem for a team of robots capable of taking range and/or bearing measurements toward a mov-ing target. They consider maximum speed and minimum sensing range constraints and plan for the next best sens-ing location using the trace of the posterior covariance matrix as the uncertainty measure.

What differentiates our problem from these works is that each measurement in our system takes a long time and the uncertainty in measurements is considerably larger. We address these factors in our strategy by using the worst-case behavior as our objective function and limiting the number of measurements as part of our planning process.

System DescriptionOur system consists of an autonomous robotic boat with onboard sensors, radio tags and receiver, and a directional antenna.

Figure 1. (a) Fish biologists manually tracking carp. (b) Targeted removal of the carp aggregation in Lake Lucy, Minnesota, using a large under-ice seine. Over 95% of all carp in this lake were captured in 4 h. (c) Robotic boat used in coverage experiments at Lake Keller, Minnesota. A directional loop radio antenna can be seen mounted on a pan-tilt unit on the top of the boat. (Photos courtesy of Peter Sorensen.)

(a) (b) (c)

During winters, the same

system is used on ground

robots operating on

frozen lakes.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Robotic BoatThe hull of our robotic platform (Figure 1) is the QBoat designed by Oceanscience (http://www.oceanscience.com/). The QBoat has dimensions 182 cm # 71 cm and can carry a payload of approximately 40 kg. The boat is capable of a maximum speed of 1.65 m/s. An onboard 12-V, 30-Ah NiMH battery allows approximately 2 h of continuous operation.

The electronics on board the boat (Figure 2) consist of a laptop capable of running high-level software, an Atmel microcontroller board for low-level interface, radio receiver equipment (described in the following section), a digital com-pass, and a Garmin 18x GPS unit. We also have a remote override radio control system that can directly control the boat, if desired.

We have a modular software architecture based on the robot operating system (ROS), comprising packages for navigation, localization, simulation, and reading of sensor data. The ROS allows remote monitoring of data from another computer on the shore via an ad hoc network formed with the onboard lap-top. At the core of the navigation package is the implementa-tion of a waypoint navigation algorithm, reported in our previous work [2], with an extended Kalman filter (EKF)-based localization routine similar to the catamaran solution presented in [14]. The electronics and software can be easily transferred to other platforms. In fact, in the winter time, we move the sys-tem to a ground robot used on frozen lakes [4].

Radio Tag and ReceiversTo sense the fish, we use radio tags manufactured by Advanced Telemetry Systems (ATS) (http://atstrack.com). A complete fish sensing system by ATS consists of radio tags, a loop antenna connected to a radio receiver, and a data logger that provides the computer interface for the receiver. Each radio tag emits a short pulse roughly once per second. The radio antenna [shown on the top of the boat in Figure 1(c)] is used to detect these pulses.

The radio receiver reports the received signal strength of the pulse. However, the signal strength is not directly useful in determining the distance to the tag, as it also depends on

the unknown depth of the fish, the conductivity of the water, and the remaining battery life of the tag. Therefore, we rely only on the directional nature of the antenna and obtain a bearing measurement toward the tag. Our method for estimating the bearing is presented in the “Field Experi-ments” section.

The tag on each fish is assigned a unique frequency. The receiver can be programmed to tune in on one or more frequencies. In the search phase, we program the receiver to loop through a list of frequencies of tagged fish in the lake. To reliably detect a pulse from a tag, the receiver needs to stay tuned to the correspond-ing frequency for more than 1 s since the tags emit pulses at about 1 Hz. After detecting a radio-tagged fish in the search phase, we program the receiver to tune in only to the corre-sponding frequency and switch to the localization phase. In the search phase, described in the next section, we do not rotate the antenna or obtain bearing measurements, since the goal of this phase is only to detect the presence of a tag.

Search and CoverageThe first phase of our monitoring task is to search for all tagged fish in the lake. One of the current models for carp mobility suggests that each fish chooses a preferred region of the lake and remains there for long periods throughout the day [3]. To increase the efficiency of our system, we restrict the search to those regions of the lake that are likely to contain the fish (Figure 3). We

R2

R3

R1

Figure 3. Incorporating domain knowledge to restrict the search region for the fish to a given set of regions, e.g.,

{ , , , } .R R R R1 2 nf= The simple approach of discretizing these regions and finding a TSP tour becomes infeasible when the regions are large.

The task of locating

tagged fish is divided into

two phases: search with

radio antenna and active

localization with bearing

measurements.

Microcontroller

Compass

Battery

Receiver

GPSMotor

Controller

Laptop

Figure 2. Top view of the electronics compartment: onboard electronics comprises a laptop, GPS, microcontroller board, batteries, digital compass, motor controller, and radio receiver.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


___

http://www.qmags.com/clickthrough.asp?url=http://atstrack.com&id=18410&adid=P35E2

http://www.qmags.com/clickthrough.asp?url=http://www.oceanscience.com/&id=18410&adid=P35E1






assume that these regions are connected in the sense that there is a path between any two points. We also assume that the fish remain stationary for the duration of covering a region, which reduces the search problem to a coverage problem.

The radio antenna has a limited sensing range. A given robot path is said to cover a point if the point lies within the sensing range of the robot at some instance along the path. The coverage problem can be defined as follows: Given a set

of connected regions { , , , },R R R R2 n1 f= find

a minimum length tour that covers every point in each region .R Ri !

A possible approach to solving this problem is based on the traveling salesperson problem (TSP). We can discretize each region with a resolu-tion dependent on the sensing range (Figure 3)

and compute a TSP tour of this set of points. However, approximation algorithms for TSP usually require a metric graph, which is typically represented as a complete graph whose vertex set is the point set to be covered. In such a rep-resentation, the number of edges is quadratic in the number of points. As the lake size or the sampling granularity increases, maintaining and operating on this large graph can become infeasible.

The coverage problem defined above is a generalization of Euclidean TSP and, consequently, nondeterministic polyno-mial-time hard (NP-hard). Next, we present a general approach for solving the coverage problem and show that the length of the path using this approach does not deviate signif-icantly from the length of an optimal tour.

Algorithm DescriptionOur general approach is composed of two steps. First, we compute a tour Rx that visits all the regions in R exactly once. We say that region Ri is visited if any point in Ri is visited by the tour. The tour Rx imposes an ordering on the regions, and defines (possibly the same) entry and exit points for each region. The entry and exit points are where Rx intersects a region for the first and the last time, respectively. Such a tour is not necessarily a solution to the original problem, since it is not guaranteed to cover all points in each region. We compute a coverage tour CRi for each region R Ri ! independently, starting and finishing at the entry and exit points for each Ri . The final tour x is constructed by augmenting the coverage tours of each region to the visiting tour Rx .

We now analyze the performance of this algorithm. Let OPT be an optimal tour that visits and covers all the regions in R in minimum time. Let Rx

) be the optimal tour that visits all the regions in R. Since OPT also visits all the regions in R,

we have ,OPT R$ x) where x denotes the length of tour x . Let CRi

) be the optimal coverage tour for a region Ri . OPTcovers every region in R : therefore, we have

.OPT CR R Ri i$ / )!

Suppose we use an α-approximation algorithm for com-puting Rx and a b-approximation algorithm for finding the coverage tour of each region. Then the tour x obtained by visiting the regions according to the order given by Rx and covering each region independently when it is visited has acost of at most .CR R R Ri i/a x b+) )

! Equivalently,

( ) .OPT` #x a b+

C OPT OPTR RR R

i

i

# #x a x b a b+ +) )

!

/

Therefore, this approach costs at most a factor ( )a b+ of an optimal algorithm.

We now present algorithms for the following two com-ponents of the strategy: computing a tour that visits the regions and covering the regions with specified entry and exit points.

Visiting the Regions: TSP with Neighborhoods and the Zookeeper ProblemsComputing a tour Rx that visits all the regions depends on the geometric properties of the regions. This problem is com-monly known as TSP with neighborhoods (TSPN). Most geo-metric instances of the TSPN problem are NP-hard. In general, we can use constant-factor approximation algorithms for TSPN such as [15] to find Rx and a .

In our application, it is reasonable to model the lake as a simply connected region, i.e., without any holes. Furthermore, more regions of interest where the fish are located are usually close to the shore. If the regions are convex polygons touching the boundary of a simply connected lake, the tour can be computed using the so-called zookeeper’s route [16]. This special case of the TSPN can be solved optimally 1a =^ h in polynomial time due to the following lemma.

Lemma 1 ([16]): Let { , , ..., , ..., }R R R R R2 i n1= be a set of convex regions located along the perimeter of a simply con-nected polygon P. An optimal solution exists for visiting the regions in R, which visits them in the order they appear along the boundary of P.

Once the ordering of the regions is known, the shortest tour visiting all regions can be calculated using dynamic programing. The exact solution is given in [16]. We use a simpler solution by discretizing the boundary of the regions to determine the entry and exit locations for each region. We build a table ( , )C i si , which stores the length of a tour that enters the region Ri at location si for the first time. The entries of the table are computed using the following recurrence:

( , ) ( ( , )) ( , ) ,min minC i s C i s d t s1i t Si i i1 1

i i1 1= - +- -

- -8 B (1)

where s 1i- and t 1i- lie on the boundary of R 1i- , and ( , )d x yis the Euclidean distance between points x and .y The cost of

Instead of using a fixed

path for k measurements,

an adaptive algorithm

is used to choose

measurement locations.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







entering the region Ri at point si is equal to the minimum cost of reaching the previous region, R 1i- , entering at location s 1i- , plus the shortest distance from R 1i- to , ( , ) .R d t s1i i i- By Lemma 1, the ordering of the regions is optimal. Since we cover all possible values of t and ,s the algorithm computes an optimal solution up to the discretization error.

To turn these tours into coverage paths, we need a way to cover a region with specified entry and exit points. We next present such a technique in which the regions are arbitrarily oriented rectangles. Rectangles are both easy to specify and general enough for practical purposes.

Covering Regions with Given Entry and Exit PointsThe algorithm presented in the subsection about optimi-zation of robot motion generates an entry and exit point for each region. These points impose a constraint on our algorithm for finding a path that covers the rectangle. The following lemma shows that we can cover a rectan-gle satisfying this constraint and be only a constant fac-tor away from an optimal coverage path without such constraints. We assume that the rectangle has an x y#grid imposed on it, such that visiting all grid cells covers the rectangle.

Lemma 2: Let R be a rectangle with a grid imposed on it. Let s and t be two grid points on the boundary specified as entry and exit points. There exists a tour T that starts at s, vis-its every grid point, and exits at t such that the length of T is at most twice that of an optimal tour that visits every grid point but can start and end at any points on the boundary of R, not necessarily s and t .

Proof: Let r be the optimal path to cover R without any restrictions on the starting and ending points. When R is a rectangle, r is a boustrophedon path, which visits every point exactly once.

Suppose that r starts at a and ends at b. Note that ,s t ! r.Without loss of generality, we assume that t is between s and balong r (see Figure 4). We form a coverage path from s to tusing r as follows: From s, go to a along r, and come back to sby retracing these steps. Then go to b from s along r (passing through t). Finally, arrive at t from b along r. This path visits every point on r and has length of at most twice that of .r

The result is tight; when the input is a rectangle with length equal to r , and ,s t= each point is covered twice.

To summarize, we showed that the following algorithm for covering rectangles along the boundary of a lake is an ( )a b+ -approximate algorithm with 1a = and .2b =

1) Compute the shortest tour Rx , which visits each region in R using the dynamic programming solution pre-sented in the section “Optimization of Robot Motion.” This algorithm returns an entry and exit point for each rectangle along with their ordering.

2) Follow Rx as follows: Starting from the entry point of an arbitrary region, whenever a region is visited, cover it using the strategy given in Lemma 2 which ends at the exit point. Move to the entry point of the next region given by Rx and repeat until all regions are visited.

ExperimentsWe implemented this search algorithm on our robotic boat and evaluated it through field trials at Lake Phalen, Minnesota. The input regions for one such trial are shown in Figure 5(a), which was chosen arbitrarily for testing our algorithm. The series of offline waypoints generated by our algorithm are shown in Figure 5(b). We used an empirically determined sensing range of 50 m to generate the boustrophedon paths. The actual trajectory executed by the robot is shown in Figure 5(c). The robot traveled a total distance of 5.6 km in about 87 min while executing this trajectory.

While moving along the search path at certain locations, the robot detected signals from five radio-tagged fish and a reference tag present in the lake. Such robot loca-tions are marked with the corresponding tag fre-quency in Figure 5(c). The actual position of the fish can be anywhere within a distance equal to the sensing range from these locations. To better localize the fish, we switch to the localization phase whenever we detect a signal on one of the tuned fre-quencies. We describe our algorithms for the localization phase next.

LocalizationThe objective of the localization phase is to use bearing measurements from the radio antenna to localize a tagged fish accurately once it is found during the search phase. The boat must choose sensing positions that provide the most information about the location of the tag. We use an EKF to estimate the position of the tag, and represent the uncer-tainty in the position of the tag with its covariance. We seek sensing locations that minimize the determinant of the covariance matrix.

s

ta

b

r

Figure 4. Covering a rectangle with given entry and exit points (s and t) with a two approximation. Follow r from s to a,complete r , then follow r from b to t. In the worst case, the optimal path r is covered twice.

Since each bearing

measurement takes about

1 min, a small number of

measurement locations

must be carefully chosen.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Measurement ModelThe received signal strength varies with the relative angle of the plane of the loop antenna with the tag. The signal is strongest when the tag is directly aligned with this plane. Since the antenna is mounted on a pan-tilt unit, we can rotate it and sample the signal strength as a function of the relative angle from the boat. We fit a smooth function to

the samples obtained and use the point of maximum value of this function as the bearing measurement. Based on a number of trials, we concluded that least squares fitting of a cubic polynomial works best for computing the bearing in our system.

Note that the bearing obtained is an infinite line (as opposed to a directed ray), and hence there is ambiguity in the obtained bearing. For example, if a is the direction with maximum signal strength, then a and a r+ are both valid bearing measurements. We disambiguate by moving along either a or a r+ and checking whether the signal strength increases or decreases. A detailed descrip-tion of other methods for disambiguating the measure-ments is given in [17].

Optimization of Robot MotionSince measuring the bearing takes time (about 1 min), the estimation must be performed using a small number, say k,of measurements. Furthermore, these locations must be cho-sen in an online fashion as the measurements become avail-able. We present three strategies to compute k sensing locations and compare their performance in simulations and real-world experiments.

All three active localization strategies require an initial estimate of the tag. In [4], we presented a scheme to initial-ize the target based on two bearing measurements taken from different sensing locations. Using this initial estimate, we propose the following three strategies to determine the next k sensing locations of the robot.

FIMThe Cramer–Rao lower bound (CRLB) for an unbiased esti-mator is a lower bound on the estimation error covariance. This lower bound is equal to the inverse of the FIM (denoted by I ) for the k measurements. The determinant of I is inversely proportional to the square of the area of the 1-vuncertainty ellipse, and is commonly used as the objective function to be maximized. For k bearing measurements with zero-mean Gaussian noise, the determinant of I (denoted by I ) is given as

( ),

sinI d d1

4 i j

i j

j

k

i

k

11

2

vi i

=-

==

; E// (2)

where ii and di are the angle and distance from the ith sens-ing location to the true target location.

We impose a grid of size n n# centered at the current position of the robot. To compute the k sensing locations, we exhaustively consider each of the ( )k

n2 combinations as a can-

didate trajectory, and compute I . An optimal trajectory can then be chosen as one with the minimum value of I .

GreedyInstead of computing a fixed path for the k measurements, we can use an online greedy strategy, which picks the next sensing location based on the current estimate and uncertainty of the position of the tag. Given the current robot and tag estimates,

-100 0 100 200 300 400 500 600

-500

-400

-300

-200

-100

0

Coverage Path

X(m)

Y(m

)

Input RegionsCoverage PathCoverageWaypoints

(b)

Robot Path with Nonzero Radio Signals by Frequency

-200 -100 0 100 200 300 400 500

-500

-400

-300

-200

-100

0

X(m)

(c)

Y(m

)

Path 48111 4827148351 48471 4875149691

(a)

R1

A

F

G

H

E O

N

I

L

J

K

M

P

B

C

D

R2

R4

R3

Figure 5. The coverage experiment conducted at Lake Phalen, Minnesota: (a) the four input regions, (b) the path found by the algorithm described in the “Localization” section, and (c) the actual path followed by the robot during coverage. The robot traveled a total distance of 5.6 km in about 87 min. Locations where signals from radio tags were detected are also marked along with their frequencies.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







the greedy strategy considers all neighboring locations of the robot as candidate-sensing locations and computes the poste-rior covariance by simulating an EKF update at each sensing location with a discretized set of possible bearing measure-ments. The greedy strategy then picks the candidate location where the determinant of the posterior is minimum.

Enumeration TreeWe extend the objective function of the greedy strategy to look ahead k measurements, in the enumeration tree strategy. We build a min-max tree that explores the set of all sensing locations and all possible measurements that can be obtained, since the uncertainty depends on both. The tree consists of two types of nodes at alternate levels (Figure 6): MAX nodes ( )ui represent neighboring robot locations to the current, and MIN nodes ( )zi represent the discretized set of possible mea-surements. Each node stores an estimate of the target’s state and covariance by simulating EKF updates based on the sens-ing locations and bearing measurements stored along the path in the tree. Details are presented in [4].

Once the tree is built, the min-max values for each node are propagated bottom-up starting with the leaf. The min-

max value for the leaf nodes is defined as the determinant of the simulated posterior covariance matrix stored at that node. The min-max value for all MAX nodes is the MAX of min-max values of its children, and that for nonleaf MIN nodes is MIN of min-max values of its children.

During execution, the robot chooses the MAX node with the minimum min-max value as the next sensing loca-tion at each iteration. The MIN node is chosen as per the actual bearing obtained. Since we use discrete measure-ment samples, there might not be a node with bearing exactly equal to the actual measurement. In addition, there is uncertainty associated with the position of the robot itself. Hence, we use the Bhattacharya Distance [18] to find a MIN node with posterior covariance closest to the covari-ance after the measurement update.

Simulations and ExperimentsWe first compared the performance of the three active localization strategies in simulation. We ran 100 random trials with the true tag 25 m away from the starting position of the robot in each trial. A grid side length 3 m was used to gener-ate sensing locations for three measurements. We generated noisy bearing measurements by corrupt-ing the true bearing with Gaussian noise ( )15cv = .

The mean errors for FIM, greedy, and enumer-ation tree were 6.30, 5.98, and 5.73 m, and the mean determinant of the final covariances were 54.81, 40.59, and 48.36 units, respectively. The poor performance for the FIM strategy can be attrib-uted to the fact that it is an open-loop strategy that depends on the initial estimate. Furthermore, it computes locations that minimize the lower bound on final uncertainty of an efficient estimator (i.e., an estimator whose variance is equal to the CRLB). Since EKF is not an efficient filter, there is no guarantee that it would achieve this lower bound. On the other hand, the enumeration tree and the greedy strategy compute the actual covariance of the EKF estimator and pick the location that would minimize its determinant.

Although the enumeration tree performs better than the greedy strategy, the performance gains are not significant to warrant the extra computational time. Hence, we decided to use the greedy strategy on our system in field experiments. To test the system, we conducted field trials with a reference tag submerged in a lake at a known position. The results from one such trial are shown in Figure 7. Sensing locations

, ,r r4 5 and r6 were obtained by running the greedy strategy. The resulting 1 v- uncertainty ellipses are shown (in blue) along with the tag estimates (shown as red crosses). The true location of the tag is marked by a black star. The final error of this triangulation was 1.21 m, with 1 v- bounds of 3.3 and 2.7 m in the x- and y-directions, respectively.

The coverage algorithm

finds a path with length

at most a constant factor

of the optimal algorithm.

-40 -30 -20 -10 0 10 20 30

-40

-30

-20

-10

0

10

20r1

r2

r3

r6

r4

r5

X(m)

Y(m

)

Localization Experiment

Figure 7. Localization experiments with the greedy strategy. Ellipses shown encompass the 1 v- uncertainty after each measurement. We use the second measurement to disambiguate which side the tag lies from bearing obtained at r1 . Bearing measurements are shown as solid green lines.

MIN

......

..... .....

......

NeigboringLocations

CandidateMeasurementsMIN

MAX

u1

z1 z12 z1 z12

u2u8

u1 u8 u1 u8Leaf

Figure 6. Min-max tree: , ,u u1 8f are the neighboring locations for the robot, and , ,z z1 12f are candidate bearing measurements.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The field experiments demonstrate that our system is capable of reliably localizing stationary reference tags. Next, we present complete field experiments in which the robot executed both the search and localization phases.

Field ExperimentsIn this section, we report results from two field tests con-

ducted at Lake Gervais, Minnesota. In the first test, a reference tag was deployed at a known location. In the second test, we searched for tagged fish in the lake.

Figure 8 shows the results from the first experiment with a refer-ence tag deployed at the location marked in Fig-ure 8(b). The robot exe-

cuted the coverage pattern while continuously monitoring

the radio antenna on the frequency of this reference tag. After detecting the signal from this radio tag at r1 [Figure 8(c)], the robot switched from the search phase to the local-ization phase.

During the localization phase, the robot executed the greedy strategy with k 2= measurements, in addition to the two initialization measurements taken at r1 and .r3 The mea-surement at r2 is used to distinguish on which side of the bear-ing at r1 the fish is located by comparing their signal strengths. The robot then moved to r4 and r5 and obtained bearing mea-surements as shown. The 1 v- uncertainty ellipse after each step is also shown. After completing the localization, the robot continued to cover the rest of the regions. The robot covered a total path of approximately 2 km in 49 min.

The second field trial (Figure 9) was conducted in the same lake without a reference tag. We programmed the robot to search for frequencies corresponding to actual radio-tagged fish in this lake. While searching the first region, the robot detected one of the frequencies in the list, executed the localization strategy, and returned to the search plan. Figure 9(b) shows the coverage path followed

With no prior information,

the robot was able to

locate the reference tag

precisely; the final error

was approximately 1 m.

r1 r2r7

r3r5

r6r4

(b)(a) (c)

-100 0 100 200-150

-100

-50

0

50

100

Start

Coverage

X(m)

Y(m

)

10 20 30 40 50

-65

-60

-55

-50

-45

-40

-35

X(m)

Y(m

)

Triangulation

R2

R1

R4

R3

F G

E

A

B

D

C

HI

J

M

P

O

N

K

L

Figure 9. Field experiments at Lake Gervais, Minnesota. (a) Areas we wish to cover. (b) Tracks computed search path. (c) Trajectory of the robots. Upon detecting a tag, the robot executed a localization strategy as illustrated. Because the robot attempted to triangulate a tagged fish, we do not know the true location of the tag.

-100 0 100 200

-100

-50

0

50

100

Start

Coverage

X(m)

Y(m

)

-10 0 10 20-40

-35

-30

-25

-20

-15

r4

r3

r1

r2

r5

r6

X(m)

(b)(a) (c)

Y(m

)Triangulation

R2F G

HE

A

B C

DI L

J K

M

PO

N

R1R3

R4

Figure 8. (a) A field trial on Lake Gervais, Minnesota. (b) The robot covered the regions via the path. (c) A close-up view of the localization of a reference tag is shown. The black star indicates the true location of the reference tag. The red box in (b) corresponds to the triangulation area (c).


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







by the robot. The red box marks the area where the robot followed the localization strategy to obtain additional bearing measurements and localize the unknown tag. Fig-ure 9(c) shows a close-up of the localization phase. Since this was an actual radio-tagged fish, the ground truth is unknown.

ConclusionsWe presented a novel system for monitoring radio-tagged invasive fish. We divided the monitoring task into two sub-tasks of finding the tagged fish and localizing them accu-rately. For the first task, we presented an algorithm for finding a tour whose length was at most a constant factor away from an optimal tour. For localizing the tagged fish, we first showed how the bearing of the tag could be esti-mated by using measurements obtained by rotating a direc-tional antenna. We then addressed the problem of actively choosing sensing locations to reduce localization uncer-tainty. We compared three algorithms in simulations and field experiments and incorporated the most effective one into our system. We concluded this article with additional field trials.

While the initial results are encouraging, there is signifi-cant future work required, including developing fish mobil-ity models and the corresponding search and tracking algorithms. We plan to use multiple boats to improve the coverage time and localization uncertainty. Additional issues faced when building such a system (e.g., communica-tion and coordination) must be addressed. We ultimately plan to use our system in larger lakes to help biologists study carp behavior.

AcknowledgmentsThe authors would like to thank members of the Sorensen Lab at the Department of Fisheries, University of Minne-sota, for engaging in many useful discussions and sharing their equipment. We also thank Deepak Bhadauria for his contributions to an earlier version of this system. This material is based upon work supported by the National Sci-ence Foundation under Grant Nos. 1111638, 0916209, 0917676, 0936710.

References[1] P. G. Bajer, C. J. Chizinski, and P. W. Sorensen, “Using the Judas technique to locate and remove wintertime aggregations of invasive common carp,” Fish-eries Managem. Ecol., vol. 18, no. 6, pp. 497–505, 2011.[2] P. Tokekar, D. Bhadauria, A. Studenski, and V. Isler, “A robotic system for monitoring carp in Minnesota lakes,” J. Field Robot., vol. 27, no. 6, pp. 779–789,2010.[3] P. Bajer, H. Lim, M. Travaline, B. Miller, and P. Sorensen, “Cognitive aspects of food searching behavior in free-ranging wild common carp,” Envi-ron. Biol. Fishes, vol. 88, no. 3, pp. 295–300, 2010.[4] P. Tokekar, J. Vander Hook, and V. Isler, “Active target localization for bearing-based robotic telemetry,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2011, pp. 488–493.

[5] J. Das, F. Py, T. Maughan, T. O, Reilly, M. Messi, J. Ryan, K. Rajan, and G.Sukhatme, “Simultaneous tracking and sampling of dynamic oceanographic features with autonomous underwater vehicles and lagrangian drifters,” in Int. Symp. Experimental Robotics, 2010.[6] R. N. Smith, J. Das, H. Heidarsson, A. A. Pereira, F. Arrichiello, I.Cetinic, L. Darjany, M.-E. Garneau, M. D. Howard, C. Oberg, M. Ragan, E.Seubert, E. C. Smith, B. Stauffer, A. Schnetzer, G. Toro- Farmer, D. A. Caron, B. H. Jones, and G. S. Sukhatme, “USC CINAPS builds bridges: Observing and monitoring the southern California bight,” IEEE Robot. Autom. Mag., vol. 17, no. 1, pp. 20–30, Mar. 2010.[7] M. Dunbabin, P. Corke, I. Vasilescu, and D. Rus, “Experiments with coopera-tive control of underwater robots,” Int. J.Robot. Res., vol. 28, no. 6, p. 815, 2009.[8] M. Dunbabin and L. Marques, “Robots for environmental monitoring: Significant advancements and applications,” IEEE Robot. Autom. Mag., vol. 19, no. 1, pp. 24–39, Mar. 2012.[9] H. Choset, “Coverage for robotics: A survey of recent results,” Annals Math. Artificial Intell., vol. 31, nos. 1–4, pp. 113–126, 2001.[10] V. Vazirani, Approximation Algorithms. New York: Springer-Verlag, 2001.[11] S. E. Hammel, P. T. Liu, E. J. Hilliard, and K. F. Gong, “Optimal observer motion for localization with bearing measurements,” Comput. Math. Appl,vol. 18, nos. 1–3, pp. 171–180, 1989.[12] E. Frew, “Observer trajectory generation for target-motion estimation using monocular vision,” Ph.D. dissertation, Stanford Univ., Stanford, CA, 2003.[13] K. Zhou and S. Roumeliotis, “Multi-robot active target tracking with com-binations of relative observations,” IEEE Trans. Robot., vol. 27, no. 4, pp. 678–695, Aug. 2011.[14] M. Caccia, M. Bibuli, R. Bono, and G. Bruzzone, “Basic navigation, guid-ance and control of an unmanned surface vehicle,” Auton. Robot., vol. 25, no. 4, pp. 349–365, Aug. 2008.[15] J. S. Mitchell, “A constant-factor approximation algorithm for TSP with pairwise-disjoint connected neighborhoods in the plane,” in Proc. 2010 Annual Symp. Computational Geometry, New York, pp. 183–191.[16] W. Chin and S. Ntafos, “The zookeeper route problem,” Inform. Sci.: Int. J., vol. 63, no. 3, pp. 245–259, 1992.[17] J. Vander Hook, P. Tokekar, E. Branson, P. Bajer, P. Sorensen, and V. Isler, “Local-search strategy for active localization of multiple invasive fish,” in Experi-mental Robotics, Springer Tracts on Advanced Robotics 88, 2012, pp. 859–873.[18] A. Bhattacharyya, “On a measure of divergence between two multinomial populations,” Sankhya: Indian J. Stat. (1933-1960), vol. 7, no. 4, pp. 401–406, 1946.

Pratap Tokekar, University of Minnesota, USA. E-mail: [email protected].

Elliot Branson, University of Minnesota, USA. E-mail: [email protected].

Joshua Vander Hook, University of Minnesota, USA. E-mail: [email protected].

Volkan Isler, University of Minnesota, USA. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_____________

______________

______________

____

_______











In crisis situations, military operating units require a rapid evaluation of the local meteorological and oceanographic (METOC) conditions affecting their missions. An important role of military oceanography (MILOC) is thus to provide a timely METOC characterization of denied

littoral areas [1]. Environmental sampling procedures in MILOC must be easily relocatable, discreet, and secure. Until recently, marine sampling technologies meeting these requirements were scarce. Remote sensing is the technology mostly used by navies to assess environmental conditions in restricted areas [2]. Very-near-shore bathymetry, sea state, surface currents, ocean color, and sea surface temperature (SST) are among the pieces of environmental information that can be obtained by means of remote sensors. Although valuable, this information is not sufficient to fully assess the three-dimensional (3-D) variability of the environment.

Numerical approaches that simulate the ocean dynamics may provide additional information on the environmental conditions. In the coastal regions, numerical ocean models of different spatiotemporal resolutions are generally nested to downscale METOC information to the region of interest [3]. Numerical procedures used to feed back dynamical information between the models with different resolutions inevitably introduce errors in this nesting process. In addition, at present, the uncertainties in physical parameterizations, forcing, and model initialization limit the accuracy of model forecasts.

Today, technological capabilities could support a new methodology to rapidly characterize marine environments in restricted areas. This methodology relies on massive in situ observations carried out by coordinated fleets of autonomous robotic platforms, specially designed for real-time observa-tion of the ocean environment, complemented by remote sensing systems. Underwater gliders, which are autonomous vehicles designed to observe vast areas of the ocean interior [4], are among these platforms. Gliders make use of buoyancy

By Alberto Alvarez, Jacopo Chiggiato, and Baptiste Mourre

Date of publication: 10 June 2013Digital Object Identifier 10.1109/MRA.2012.2220505

Rapid Characterization of Restricted Marine Environments

Under the Sea

PH

OTO

CO

UR

TE

SY

OF

U.S

. NA

VY

(B

AC

KG

RO

UN

D);

A A

LVA

RE

Z, J

. CH

IGG

IATO

, AN

D B

. MO

UR

E (

GLI

DE

R)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







changes, their hydrodynamic shape, and small wings to carry out undulatory motions between the surface and a predeter-mined depth with a net horizontal displacement. This propul-sion procedure requires very low energy consumption and provides gliders with up to several months of autonomy at sea. This capability has raised a growing interest in glider technology of naval forces. For example, the U.S. Navy recently awarded a contract to the glider manufacturer Webb Research Corporation-Teledyne Brown Engineering Inc. to provide a fleet of up to 150 glider vehicles. The fleet would be operated for persistent surveillance and monitoring from the T-AGS60 Pathfinder survey ships [5].

The advent of glider technology to assess environmental conditions creates new scientific and technological demands, e.g., the proper exploitation of the information content of the data collected by this kind of platform. The oceanographic data is often exploited by representing the sampled field on a regular grid to facilitate the extraction of dynamic information from the data. The two most common interpolation methods found in oceanographic and meteorological literature are the best linear prediction scheme [(BLP), “kriging,” or “objective analysis”] [6], [7] and spline interpolation [8]. The former scheme is commonly used in different scientific disciplines to assign, from the data gathered at arbitrary locations, the best values at grid points of a regular grid [6]. This approach relies on a priori knowledge of the mean and covariance of the sam-pled field to provide the best linear estimation of the average and variance of the field at given unsampled locations. Unfor-tunately, knowledge of a covariance model is problematic in regions such as coastal areas, where historical data may be sparse or even nonexistent [9], [10]. Moreover, gliders generate spatially dense measurements, which may cause this approach to be computationally unfeasible.

Spline models are found in meteorological literature as an alternative to the estimation scheme discussed above [8]. From a stochastic point of view, the spline model technique provides the maximum-likelihood estimate from the data and a priori information that the first (membrane model) or sec-ond derivatives (plate model) are zero everywhere and the result of random errors, i.e., white noise. In other words, it provides the most probable continuous (membrane model) or differentiable (plate model) field compatible with observa-tions. BLP and spline interpolation methods are reviewed and compared in [11]. It is also concluded in [11] that spline methods are preferred over BLP schemes when the underly-ing statistics of the spatiotemporal variability of the area are poorly known, which is the case of the marine region consid-ered in this article. In addition to the above schemes, a review of other data interpolation methods less common in oceanog-raphy can be found in [12].

Spline models have recently been proposed to reconstruct underwater thermal fields from sparse spatially biased data gathered by an autonomous underwater vehicle (AUV) and remote sensing [12]. Specifically, the variational formulation underlying the spline procedure was solved using a finite ele-ment technique to substantially reduce the dimensionality of

the problem, and the remote sensing was incorporated as a Dirichlet boundary condition. The approach resulted in reli-able estimations of the volumetric variability of the underwa-ter thermal field in a small coastal area off the coast of Latvia (Baltic Sea), where an AUV recorded temperature data while performing a mission to search for sea mines. This field experiment, conducted by the North Atlantic Treaty Organi-zation (NATO) Undersea Research Centre (NURC) in April 2009, provided a valuable data set to test and compare differ-ent procedures to integrate data from remote sensing and underwater robots into a unified environmental picture. Unfortunately, the small size of the area of operations and the use of a single robotic platform limited the scientific exploita-tion of this field experiment. A more comprehensive field experiment was carried out by the same institution a year later in the western Mediterranean to further investigate the capability of a fleet of gliders to characterize a spatially extended restricted area when supported by remote sensing data. Based on this field experiment, this article compares the environmental assessments obtained from 1) the spline mod-els and 2) a traditional modeling approach in their ability to represent the temperature field.

The Rapid Environmental Picture 2010 Field ExperimentA field experiment named Rapid Environmental Picture 2010 (REP10) was conducted by the NURC with the NATO Research Vessel Alliance during August 2010 in an ocean region off of La Spezia, Italy, in the Ligurian Sea. A nearly rectangular and access-restricted area was defined with dimensions of approximately 60 km × 80 km, as illustrated in

Figure 1. Geographical location of the REP10 field experiment. This experiment aimed to characterize a prescribed access-restricted area (gray polygon) by using observations from three Slocum gliders (thick black lines) and remote sensing SST (colored field in °C, here for 21 August 2010). A second data set was collected by a towed ScanFish for validation proposes (red lines). The brown dashed arrow represents the main stream of the NC in the region. Finally, contours of the 50-, 200-, 500-, 1,000-, and 2,000-m isobaths are illustrated by the white lines. The inset shows the position of the experimental area in the Western Mediterranean basin.

44

43.8

43.6

43.4

43.2

43

42.8

42.6

42.4

8

1,000

2,000

50

1,000

200

500

500

50050

50

50200

200

8.5 9 9.5

Longitude E

Corsica

Italy

LaSpezia

NC

10 10.5 11

44.2

44.4

25

24.5

24

23.5

23

22.5

22

25.5

26

Latit

ude

N


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Figure 1. Depth values range from 50 m up to almost 1,800 m in this area. From the oceanographic point of view, this region is located in a frontal zone that divides the warmer and lighter coastal waters from the colder and denser waters of the cen-tral zone [13]. Although this structure is rather persistent, it has a significant seasonal and interannual variability and intense mesoscale activity related to the strong frontal system. A main current system, the so-called Northern Current (NC), shown in Figure 1, flows along the continental slope with speeds of 0.3–0.4 m/s−1. Synoptic time scales are in the order of three days in this region.

A fleet of three coastal Slocum gliders [14] (Figure 2) was deployed on 20 August 2010 at the southwestern boundary of the access-restricted area, aiming to characterize this hypo-thetical battlespace. Deployments were equidistant, with two gliders at the northern and southern boundaries and the last one on the center line of the region. Glider trajectories at the northern and southern boundaries were defined as straight lines toward the coast and perpendicular to the continental slope. The objective of these gliders was to characterize the boundaries by means of perpendicular cross sections of the frontal system associated to the continental slope. The central

glider was also directed toward the coast but its trajectory was modified depending on local current information. The gliders transited the access-restricted area until 22 August. The fleet collected a total of 419 vertical profiles of water conductivity (salinity), temperature, and depth (pressure) (CTD) between 20- and 180-m depth and with a horizontal interval of ~400 m (Figure 3) using an unpumped Seabird 41 CTD sen-sor with an accuracy of 0.002 °C and a resolution of 0.001 °C. The gliders surfaced every 3 h to transmit the collected data. In this work, the environmental assessment is focused on the 3-D thermal field. Characterization of the underwater tem-perature field is typically the foremost requirement in MILOC for antisubmarine warfare operations as it provides the basis for determining the range and performance of sonar systems. In addition, the spatial distribution of this quantity at the sea surface is routinely measured from remote sensing platforms. In this field experiment, in situ measurements from the fleet of gliders were merged with an ultra-high Mediterranean res-olution SST analysis map with 1-km resolution obtained on 21 August (Figure 1). This map is based on infrared measure-ments collected by radiometers on various satellites with sta-tistical interpolation to fill cloudy areas [15]. In addition, a towed vehicle, ScanFish MK II, equipped with a pumped Sea-Bird 49 CTD sensor was used to sample the restricted area on 21 and 22 August to provide sea-truth data for the validation of the environmental characterization procedures. The tem-perature resolution of the sensor is 0.0001 °C with an accu-racy of 0.002 °C. The towed vehicle undulated between 5- and 80-m depths at a horizontal speed of 3 m/s−1, following the trajectory displayed in Figure 1. The corresponding tempera-ture profiles are illustrated in Figure 4.

Robotic Marine Environmental CharacterizationThe data set collected by the robotic fleet was merged with the SST map following a similar methodology as proposed in [12] to generate an integrated picture of the volumetric tem-perature distribution in the region. Hereafter, this method will be referred to as the “data-driven assessment method.”

Figure 2. A Slocum glider used by NURC in the REP10 field experiment.

Figure 3. Temperature sections in °C between 20- and 85-m depth collected by the fleet of gliders on 21 and 22 August. The coastline is represented by the black line.

43.443.6

LatitudeLongitude

Dep

th

43.844

44.2

8.8 9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0

2423222120191817161514

25

Figure 4. Temperature profiles in °C between 5- and 80-m depth collected by the towed ScanFish on 21 and 22 August. The coastline is represented by the black line.

43.6 LatitudeLongitude

Dep

th

43.8

44

44.2

9 9.19.2 9.3 9.4 9.5 9.6 9.7 9.8

-10-20-30-40-50-60-70-80

0

2423222120191817161514

25


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The temperature measurement of each glider at a given loca-tion ( , , )x y zi i i is assumed to result from the true value of the temperature field at this location, ( , , )T x y zi i i plus an inde-pendent Gaussian noise with standard deviation ( , , ) .x y zi i iv Measurements are assumed synoptic, and thus

no time dependence is considered in the analysis. The repre-sentation error associated with the gliders and Scanfish mea-surements and accounting for the potential departure from synopticity is assumed to be 0.1 °C, as previously estimated in [16] for the same region, time period, and depth range. Under these assumptions, the probability to get the set of measurements ( , , )d x y z 1i i i i Nf=" , for a given realization of the field ( , , )T x y z is provided by the likelihood density [17]:

( , , ) ,p d T x y z e( )T d

2 i

i i

i

N

2

2

1\ v-

-

=^ h / (1)

where Ti refers to the value of the field at the ith sampling location, , , .T T x y zi i i i= ^ h According to Bayes’ rule, the pos-terior probability to have the field ( , , )T x y z given the set of observations ( , , )d x y zi i i i N1"=" , is

( , , )( )

( ( , , )) ( ( , , )),P T x y z d p d

p d T x y z p T x y z=^ h (2)

where ( )p d is the probability density of the observations, ( , , )P T x y z e ( )F T2\

a-^ h is the a priori probability of the tem-perature field, and a is a smoothing parameter determined from the data. Only the thin-plate model, ( )F T =

( , , ) ,T x y z dxdydzV

2d### is considered in this work, as it provides significantly better performance than membrane models defined by ( ) ( , , )F T T x y z dxdydz

V

2d= ### [12]. Under this consideration, the posterior probability is

( , , ) ,P T x y z d e( )

( )2T d F T

2 i

i i

i

N

2

2

1\ v

a--

-=^ h / (3)

and the maximum a posterior (MAP) estimate is defined by the field ( , , )T x y zMAP that satisfies:

( , , )( )

( ) .2

arg minT x y z T d F T2MAP

2

2

T i

i i

i

N

1 va=

-+

=

e o/ (4)

Thus ( , , )T x y zMAP is the most probable field compatible with our level of knowledge described by the smoothness constraint and the data collected by the fleet of gliders. The field ( , , )T x y zMAP can be calculated using a variational approach, where satellite data constrain the boundary values ( , , ) .T x y 0 This procedure optimizes the exploitation of

information available from remote sensors and gliders.Equation (4) is solved using a 3-D finite element approach.

The total ocean volume under consideration V is discretized as an unstructured mesh constituted by prismatic elements defined by 15 nodes [18]. In the present case, a 3-D grid of 1,319 nodes and 387 prismatic elements was generated from 0- to 85-m depth in the region of interest (Figure 5). This grid corresponds to a segmentation of the volume with ten layers of prismatic elements of 8.5-m depth and triangular faces with approximating 12-km edges. This guarantees a mini-mum vertical and horizontal resolution of 4 m and 6 km, respectively. At this resolution, the Rossby radius of deforma-tion (representing a fundamental horizontal scale of meso-scale eddies and of the order of 12 km in this region [19]) is resolved. Thus, the present discretization is appropriate to estimate the main spatial variability in the region with a lim-ited computational demand.

Following the standard finite element methodology, the value of the temperature field ( , , )T x y z inside an eth pris-matic unit of this grid is encoded by the value of the field at each node and a set of interpolation functions:

( , , ) ( , , ) ,T x y z N r s t Tk kk 1

15=

=

/ (5)

where Tk is the temperature at the kth node of the eth pris-matic element and ( , , )N r s tk are the interpolation functions expressed in a local coordinate system {r, s, t} [13]. Notice that the local coordinates in the interpolation functions are functions of the global coordinate system (x, y, z). Confining (4) to the eth prismatic element and substituting (5) into (4) results in [20], [21]

,K A gije

ije

j ieW+ =^ h (6)

with matrices given by

K

xN

xN

yN

yN

zN

zN

xN

yN

xN

zN

yN

zN

dxdydz2

2 2

2

2

2

2

2

2

2

2

2

2

2

2

ije

i j i j

i j i j

i j i jVe

2

2

2

2

2

2

2

2

2

2

2

2

22

2

2

22

2

2

22

2

2

a=

+

+ +

+ +

p

r

qqqqqqqqq

t

v

uuuuuuuuu

###

( ) ( )g N x d x2i

e

k

i

k

N

1

k ke

v=

=

/( ) ( )

,g N x d x2i

e

k

i k k

k

N

1

e

v=

=

/ (7)Figure 5. The prismatic elements used to discretize the volume of the restricted area (only the first layer is fully displayed). The coastline is represented by the black line in the upper right corner.

43.643.4 Latitude

Longitude

Dep

th

43.844

44.2

8.8 9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







where Ve and Ne are the volume and total number of sam-ples distributed inside the eth prismatic element. The above-described procedure is applied to all prismatic elements in the grid and the resulting set of equations assembled into a global stiffness matrix K Aij ij+^ h and loading vector gi [13]. The smoothing parameter a is computed from the data using the discrepancy principle [22]. The global system of equations is solved after adequately introducing the boundary conditions provided by the SST map, and it leads to temperature values at the nodes of the mesh. Thermal field estimations are then possible using (5) at any location embedded in the grid.

Model-Driven Marine Environmental CharacterizationThe model-driven approach is based on the outputs from an operational ocean forecasting system. Setting up a relocatable regional numerical ocean model that is nested in a larger scale model (now routinely available worldwide) is another option to obtain useful environmental information. The ocean model employed in this application is the Regional Ocean Modeling System (ROMS) [23]. The ROMS is a primitive equation, finite difference, hydrostatic, and free surface model. It uses generalized terrain following s-coordinates, a staggered Arakawa C grid in the horizontal, and a split-explicit, nonhomogeneous predictor/corrector time stepping. The ROMS kernel is described in detail by [24].

The model was set up in operational forecast mode dur-ing the REP10 trial framework over a domain covering the entire Ligurian Sea, with two open boundaries located on the western and southern sides (Figure 6). The horizontal resolu-tion is 2 km, with 32 vertical s-levels nonlinearly stretched to resolve the surface boundary layer. Advection for momen-tum is integrated using a third-order upstream scheme [25], whereas advection for tracers is integrated using a MPDATA family scheme [26]. A very weak grid-size-dependent har-monic form of the horizontal diffusivity is applied, while no horizontal viscosity is used (however, the third-order

upstream advection scheme for momentum contains some implicit diffusion [25]). The pressure gradient term is solved by a density Jacobian with cubic polynomial fits [27]. Param-eterization of the vertical mixing follows the generic length scale approach [28], [29].

The atmospheric model COSMO-ME of the Italian Air Force National Meteorological Center (CNMCA) provided the surface forcing for the ocean model. COSMO-ME is the 7-km CNMCA operational setup of the nonhydrostatic regional model developed by the Consortium for Small-Scale Modeling (COSMO) and based on the Lokal Modell [30]. Turbulent air–sea interaction fluxes (momentum, heat, fresh water) are calculated interactively using ROMS SST and COSMO-ME surface fields (wind, air temperature, relative humidity and mean sea-level pressure). Net longwave radia-tion flux is estimated by an internal algorithm using the total cloud cover from the atmospheric model and ROMS SST. Net shortwave radiation flux is directly provided by the atmo-spheric model.

Open boundary conditions are applied to tracers and baro-clinic velocity with radiation and nudging [31]. Nudging time scales are one day on inflow and five days on outflow. The free surface and depth-integrated velocity boundary conditions use the method of Flather [32]. External values used at the open boundaries were provided by daily averages of large-scale Mediterranean Forecasting System (MFS) forecasts [33]. The MFS assimilates sparse observational data daily available in the Mediterranean Sea. This data may include sea-level anomalies, SST, in situ temperature profiles from expendable bathyther-mographs (XBTs) from voluntary observing ships (VOS), and in situ temperature and salinity profiles from array for real-time geostrophic oceanography floats [33].

The operational ROMS-based system was initialized on 1 May 2010 using an analysis field from the MFS model. Since then, the ROMS has been run in forecast mode once a day (00:00 UTC) without data assimilation, with output data every 3 h and a forecast range of 72 h.

Evaluation of the Thermal Field EstimationsThe environmental characterizations resulting from the data- and model-based assessments are validated and compared in this section. Validation is achieved by comparing the esti-mated temperature at each location and depth sampled by the ScanFish-CTD sensor with the observed value. The overall performance of the estimation models is evaluated using the normalized explained variance

( , , ) ( )

( , , ) ( , , )

,RT x y z T z

T x y z T x y z12

2

2

i i i ii

Np

i i i i i ii

Np

1

1= -

-

-

=

=

t

^

^

h

h

/

/(8)

where ( , , )T x y zi i it and ( , , )T x y zi i i are the estimated and observed temperatures at the locations of ScanFish-CTD obser-vations, , , ,xi yi zi i

Np1=" , N p is the total number of ScanFish-

CTD observations, and ( )T z is the average profile. The skill

Figure 6. The mean ROMS surface currents and temperatures in °C on 21 and 22 August.

44

43.8

43.6

43.4

43.2

43

42.8

42.6

42.4

8 8.5 9 9.5Longitude E

Corsica

Italy

La Spezia1 m/s

10 10.5 11

44.2

44.4

25

24.5

24

23.5

23

22.5

22

25.5

26

Latit

ude

N


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







score R2 is a dimensionless coefficient used to compare the vari-ance explained by the estimated fields with the total variance of the observations considered as validation data set [34]. Values of R2 close to one indicate good performance, while values equal to or less than zero are suggestive of limited skill.

Figure 7 displays the vertical profiles estimated from the data-driven assessment method and the associated error. The estimated temperature field shows a well-defined layered verti-cal structure with an appropriate range of temperature values at depth. However, the estimated field resembles a smoothed version of the real data. The data-driven assessment method fails to represent the small-scale variability observed in the thermocline records from the ScanFish. This is evidenced by the uneven vertical distribution of errors in Figure 7(b). In fact, errors are rather low at all depths, except in the layer between 20- and 30-m depth. Temperature gradients associ-ated with the thermocline lead to significant inaccuracies if the depth and gradient characteristics of the thermocline are not

perfectly reproduced in the estimation process. The overall performance of the data-driven assessment method is given by a value of 0.51 for ,R2 indicating that half of the initial variance in the data is properly explained.

Profile estimations and corresponding errors from the model-driven strategy are displayed in Figure 8(a) and (b), respectively. Temperatures estimated in the surface layer are cooler than the ScanFish observations. The model predicts an eastward negative tilt in the thermocline depth, with warmer temperatures onshore. This corresponds to the thermal structure observed by the ScanFish in the two northernmost transects, but it is reversed when compared with the structure found along the southern transect. Com-pared with results from the data-driven assessment method, errors are larger throughout the water column, and more particularly near the thermocline. The R2 is 0.14, which indicates an overall lower skill compared with the other method.

Figure 7. (a) Temperature field estimations and (b) associated errors in °C obtained from the data-driven environmental assessment at the locations sampled by the ScanFish. The coastline is represented by the black line.

(a)

(b)

LatitudeLongitude

Dep

th

9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0

24

23

22

21

20

19

18

17

16

15

14

25

43.6

43.8

44

44.2

2

1

0

-1

-2

-3

3

LatitudeLongitude

Dep

th

9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0

43.6

43.8

44

44.2

Figure 8. (a) Temperature field estimations and (b) associated errors in °C obtained from the model-driven environmental assessment at the locations sampled by the ScanFish. The coastline is represented by the black line.

(a)

(b)

2

1

0

-1

-2

-3

3

LatitudeLongitude

Dep

th

9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0

43.6

43.8

44

44.2

LatitudeLongitude

Dep

th

9 9.2 9.4 9.6 9.8

-10-20-30-40-50-60-70-80

0

24

23

22

21

20

19

18

17

16

15

14

25

43.6

43.8

44

44.2


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The validation of both assessments is done more quantita-tively in Figure 9. This figure displays the root mean square error computed for each 5-m-thick vertical layer. Results again indicate that both estimation procedures fail to accu-rately reproduce the variability observed at the thermocline depth. The data-driven approach, based on glider and remote sensing data, outperforms the numerical model at the surface and in deeper layers. The similarity in the shapes of the curves provides insights about the nature of the error in the model-driven assessment. Except for the surfacemost layer, the ther-mal estimation obtained from the model is shifted toward warmer values. Further analysis confirms that the numerical model temperature field has a bias of 0.44 °C, whereas the data-driven estimate is essentially unbiased.

Data- or Model-Driven Marine Environmental Characterization?Observational MILOC is transitioning from ship-based capabilities to networked robotic platforms, which ensure safe, cost effective, and discreet access to areas otherwise denied to traditional units. Present glider technology is mature enough to carry out persistent environmental moni-toring by a fleet of robots, which allows a battlespace envi-ronmental characterization exclusively on the basis of observational assets. This contrasts with the traditional numerical modeling characterization. This article compared the two approaches, with the aim to quantify the gain in information, if any, obtained from a robotic environmental characterization. Results show that the data-driven environ-mental assessment substantially improves the field estima-tions compared with the model-driven environmental assessment. The environmental characterization based on the glider and remote sensing data is accurate in the surface layer and below the thermocline. Integration of remote sens-ing information is essential to improve the results at the sur-face, because no glider measurement is considered above 20 m to simulate battlespace safety conditions. Below the

thermocline, the accuracy of the data-driven estimation is related to the low spatial variability of the thermal field in the deep layers. The accuracy is substantially degraded in the thermocline. As this portion of the water column is of special interest for MILOC because of the impact of large tempera-ture gradients on the sound propagation, devoting part of the glider fleet to track the thermocline variability (e.g., confin-ing glider paths between 20- and 40-m depth) should be considered in the future.

It is certainly easier to reconstruct environmental condi-tions from observations than from the numerical simulation of the full dynamical interactions. However, despite the lower accuracy obtained in this experiment by the numerical model, environmental simulation remains a very valuable tool as it provides not only a synoptic description of the oceanic field but also a predictive view of the marine environment. Numeri-cal ocean models can now be considered mature in the simula-tion of processes like external tides, storm surges, river plumes, coastal topographic waves, upwelling and alongshore currents, and mesoscale activity [35]. Notice that the direct characteriza-tion from glider observations of part of these environmental processes is unfeasible because of the slow motion and lack of accurate underwater positioning of glider platforms. Indeed, numerical models are the core of the optimized environmental characterization discipline, focused on developing predictions of the evolution of the marine environment. Further research is required to improve the models’ accuracy. Numerical ocean models are inevitably limited in their spatiotemporal resolu-tion, which necessitates use of ad hoc parameterizations to represent the subgrid scale processes. For example, vertical mixing (including the mixing induced by internal waves) involves very small-scale processes (i.e., turbulence), and thus, it is parameterized in the present model application. This is then reflected in the simulation of the thermocline, which is too smooth when compared with observations. Such features can be better represented with robotic platforms such as glid-ers. It should be noted that no data were assimilated in the area of interest in the numerical approach considered in this study. Assimilation of observations from robotic fleets would likely improve the realistic representation of marine environments by operational ocean models. In this approach, the network topology of the robotic fleet may be established by a continu-ous feedback of information between the observational nodes and a numerical simulation engine that produces a physically consistent analysis of the battlespace on the basis of the infor-mation received from in situ and remote sensors. The hierar-chy of near future sampling strategies established by data assimilation may be communicated to the robots through underwater or above water communication networks. Exploit-ing synergism between both procedures is envisioned to improve the assessment and predictions of marine environ-ments. The performance of the above procedure (known as adaptive sampling) to characterize mesoscale activity in access-restricted marine areas, is being investigated at NURC.

To summarize, our results indicate that marine underwa-ter characterization obtained from adequate observational

Figure 9. Vertical distribution of the RMSE for the data-driven (solid line) and model-driven (dashed-dotted line)environmental assessment methods.

0 0.5 1

RMSE

Dep

th

1.5 2

-70

-60

-50

-40

-30

-20

-10

0


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







technologies and estimation methodologies is more accurate than the estimation provided by the numerical simulation of the full environmental dynamics. This observational charac-terization is made possible by the present glider technology, which provides a way for persistent in situ monitoring of restricted areas. Remote sensing complements the sparse glider sampling in the surfacemost layer. For environmental fields that are characterized by relatively long synoptic time scales such as the temperature field, the nowcast obtained from observations is also expected to provide a good predic-tion of near-future conditions (this is known as a persistence model). Yet numerical modeling remains the only available tool to characterize marine environments for longer forecast ranges. Note that although the described data-driven approach was developed for military applications, it can be applied without modification in civilian oceanography to characterize marine areas to which access may be restricted due to hazardous conditions (e.g., severe storms, oil spills, radioactive polluted areas).

AcknowledgmentsThe COSMO-ME atmospheric model data have been kindly provided by CNMCA, Rome, Italy. The MFS ocean model data have been kindly provided by INGV, Bologna, Italy. The SST Analysis map is courtesy of CNR-ISAC, Rome. This work was funded by the North Atlantic Treaty Organization.

References[1] P. G. Renaud, “In-stride battlespace characterization,” in Proc. OCEANS Conf., 2003, pp. 1752–1757.[2] C. Harris, “The omniscient eye: Satellite imagery, battlespace awareness, and the structures of the imperial gaze,” Surveillance Soc., vol. 4, nos. 1–2,pp. 101–122, 2006.[3] A. R. Robinson, J. Sellschopp, W. G. Leslie, A. Alvarez, G. Baldasserini, P. J.Haley, P. F. J. Lermusiaux, C. J. Lozano, E. Nacini, R. Onken, R. Stoner, and P.Zanasca, “Forecasting synoptic transients in the Eastern Ligurian Sea," unpublished.[4] H. Stommel, “The slocum mission,” Oceanography, vol. 2, no. 1, pp. 22–25, 1989.[5] M. Rusling, (2009, June). Gliders will aid naval research. National Defense[Online]. Available: http://www.nationaldefensemagazine.org[6] L. M. Stein, Interpolation of Spatial Data. New York: Springer-Verlag,1999, p. 246.[7] F. P. Bretherton, E. Davis, and C. B. Fandry, “A technique for objective analysis and design of oceanographic experiments applied to MODE-73,” Deep Sea Res., vol. 23, no. 1, pp. 559–582, 1976.[8] G. Whaba and J. Wendelberger, “Some new mathematical methods for var-iational objective analysis using splines and cross validation,” Mon. Weather Rev., vol. 108, no. 8, pp. 1122–1143, 1980.[9] A. Alvarez and E. Reyes, “Volumetric estimations of thermal fields inferred from glider-like and remote sensing measurements in undersampled coastal regions,” J. Geophys. Res., vol. 115, no. C11006, p. 13, 2010.[10] K. L. Denman and H. J. Freeland, “Correlation scales, objective mapping and a statistical test of geostrophy over the continental shelf,” J. Mar. Res., vol. 43, no. 3, pp. 517–539, 1985.[11] P. C. McIntosh, “Oceanographic data interpolation: Objective analysis and splines,” J. Geophys. Res., vol. 95, no. C8, pp. 13529–13541, 1990.[12] A. Alvarez, “Volumetric reconstruction of oceanographic fields estimated from remote sensing and in situ observations from AUVs of opportunity,”IEEE J. Oceanic Eng., vol. 36, no. 1, pp. 12–24, 2011.[13] M. Astraldi and G. P. Gasparini, “The seasonal characteristics of the circulation in the North Mediterranean basin and their relationship with the atmospheric-climatic conditions,” J. Geophys. Res., vol. 97, no. C6, pp. 9531–9540, 1992.

[14] D. C. Webb, P. J. Simonetti, and C. P. Jones, “Slocum: An underwater glider propelled by environmental energy,” IEEE J. Oceanic Eng., vol. 26, no. 4,pp. 447–452, 2001.[15] B. Buongiorno Nardelli, C. Tronconi, A. Pisano, and R. Santoreli, “High and ulta-high resolution processing of satellite sea surface temperature data over Southern European seas in the framework of MyOcean project,” Remote Sens. Environ., vol. 129, pp. 1–16, Feb. 2013.[16] A. Alvarez and B. Mourre, “Optimal sampling designs for a glider mooring observing network,” J. Atmos. Ocean. Technol., vol. 29, no. 4, pp. 601–612, 2012.[17] A. Tarantola, Inverse Problem Theory. Philadelphia, PA: SIAM, 2005, p. 342.[18] G. Dhatt and G. Touzot, The Finite Element Displayed. New York: Wiley,1984, p. 509.[19] F. Grilli and N. Pinardi, “The computation of Rossby radii of deformation for the Mediterranean Sea,” MTP News, no. 6, p. 4, 1998.[20] P. P. Brasseur, “A variational inverse method for the reconstruction of general circulation fields in the northern Bering Sea,” J. Geophys. Res., vol. 96, no. C3, pp. 4891–4907, 1991.[21] E. Bohnsack, “Continuous field approximation of experimentally given data by finite elements,” Comput. Struc., vol. 63, no. 6, pp. 1195–1204, 1997.[22] P. C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. Philadelphia, PA,: SIAM, 1998, p. 247.[23] D. B. Haidvogel, H. Budgell, W. P. Cornuelle, B. D. Curchitser, E. Di Lor-enzo, E. Fennel, K. Geyer, W. R. Hermann, A. J. Lanerolle, L. Levin, J. McWil-liams, J. C. Miller, A. J. Moore, A. M. Powell, T. M. Shchepetkin, A. F. Sherwood, C. R. Signell, R. P. Warner, and J. C. Wilkin, “Ocean forecasting in terrain-fol-lowing coordinates: Formulation and skill assessment of the Regional Ocean Modeling System,” J. Comput. Phys., vol. 227, no. 7, pp. 3595–3624, 2008.[24] A. F. Shchepetkin and J. C. McWilliams, “The regional ocean modelling system: A split-explicit, free-surface, topographyfollowing-coordinates ocean model,” Ocean Modell. vol. 9, no. 4, pp. 347–404, 2005.[25] A. F. Shchepetkin and J. C. McWilliams, “Quasi-monotone advection schemes based on explicit locally adaptive dissipation,” Mon. Weather Rev., vol. 126, no. 6, pp. 1541–1580, 1998.[26] L. Margolin and P. K. Smolarkiewicz. “Antidiffusive velocities for multipass donor cell advection,” SIAM J. Sci. Comput., vol. 20, no. 3, pp. 927–929, 1998.[27] A. F. Shchepetkin and J. C. McWilliams, “A method for computing hori-zontal pressure-gradient force in an oceanic model with a non-aligned vertical coordinate,” J. Geophys. Res., vol. 108, no. C3, p. 3090, 2003.[28] L. Umlauf and H. Burchard, “A generic length-scale equation for geophys-ical turbulence,” J. Mar. Res., vol. 61, no. 2, pp. 235–265, 2003.[29] J. C. Warner, C. R. Sherwood, H. G. Arango, and R. P. Signell, “Perfor-mance of four turbulence closure models implemented using a generic length scale method,” Ocean Modell., vol. 8, nos. 1–2, pp. 81–113, 2005.[30] J. Steppeler, G. Doms, U. Shatter, H. W. Bitzer, A. Gassmann, U. Damrath, and G. Gregoric, “Meso-gamma scale forecasts using the nonhydrostatic model LM. Meteorol,” Atmos. Phys., vol. 82, nos. 1–4, pp. 75–96, 2003.[31] P. Marchesiello, J. C. McWilliams, and A. F. Shchepetkin, “Open bound-ary conditions for long-term integration of regional oceanic models,” Ocean Modell., vol. 3, nos. 1–2, pp. 1–20, 2001.[32] R. A. Flather, “A tidal model of the north-west European continental shelf,” Mem. Soc. R. Sci. Liège, vol. 6, no. 6, pp. 141–164, 1976.[33] P. Oddo, M. Adani, N. Pinardi, C. Fratianni, M. Tonani, and D. Pettenuzzo, “A nested atlantic-mediterranean sea general circulation model for operational forecasting,” Ocean Sci., vol. 5, no. 4, pp. 461–473, 2009.[34] J. Schmidli, C. Schmutz, C. Frei, H. Wanner, and C. Schar, “Mesoscale precipitation variability in the region of the european alps during the 20th century,” Int. J. Clim., vol. 22, no. 9, pp. 1049–1074, 2002.[35] J. McWilliams, “Targeted coastal circulation phenomena in diagnostic analyses and forecast,” Dyn. Atmos. Oceans, vol. 48, nos. 1–3, pp. 3–15, 2009.

Alberto Alvarez, NATO Centre for Maritime Research and Experimentation (CMRE, formerly NURC), La Spezia, Italy. E-mail: [email protected].

Jacopo Chiggiato, Istituto di Scienze Marine (CNR-ISMAR), Venezia, Italy. E-mail: [email protected].

Baptiste Mourre, NATO Centre for Maritime Research and Experimentation (CMRE, formerly NURC), La Spezia, Italy. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


______________

__________________

______________

http://www.qmags.com/clickthrough.asp?url=http://www.nationaldefensemagazine.org&id=18410&adid=P49E1










Robotized Plant Probing

By Guillem Alenyà, Babette Dellen, Sergi Foix, and Carme Torras

Leaf Segmentation Utilizing Time-of-Flight Data

Supervision of long-lasting extensive botanic experiments is a promising robotic application that some recent technological advances have made feasible. Plant modeling for this application has strong demands, particularly in what concerns

three-dimensional (3-D) information gathering and speed.

Probing of Plant LeavesRecent advances in depth sensors [1], deformable object mod-eling [2], and autonomous mobile manipulation [3] have con-siderably widened the scope of robot applications. One area that is currently gaining attention and could be benefited from all these advances is the monitoring and maintenance of large

botanic experimentation fields, e.g., for plant phenotyping. The goal is to determine the best treatments (watering, nutrients, and sunlight) to optimize predefined aspects (plant growth and flowers), and, toward this aim, experiments entail-ing many repetitive actions need to be conducted [4]. Mea-surements and samples from leaves must be regularly taken, and some pruning may need to be performed [5]. These are tasks for which robots would be very handy; however, difficul-ties arise from the complex structure and deformable nature of plants, which not only change appearance by growing but whose leaves also move on a daily cycle.

In the last 20 years, several robotic systems have been introduced for the automated harvesting of tomatoes, cucumbers, mushrooms, cherries, strawberries, and other fruits (for a review see [6]), but these systems have not yet reached the stage of commercialization due to the challenges posed by the task. The automated probing of plant leaves is a


Date of publication: 27 June 2013


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







related but new research topic in agricultural robotics with many potential applications. For example, probes could be taken from plants automatically to detect plant diseases or nutritional deficiencies. The treatment of singular plants can then prevent spreading of disease in fields and reduce the application of chemicals. Another potential application is the fast probing of plants in research laboratories for phenotyping purposes. We expect to face similar challenges as the ones previously encountered with picking robots in agriculture:1) the recognition and localization of the target, e.g., fruits

and leaves, given the varying appearances of plants 2) the probing, grasping, cutting, or detachment of parts of

the plant under weakly constrained conditions in natural environments. Another major challenge in agricultural robotics is the

guidance of motions through crop fields or greenhouses, which is not addressed in this article. The first challenge requires new solutions for the recognition and localization of leaves to be developed. Previously, color vision has been used to obtain some relevant plant features, mainly for rec-ognition and classification purposes [7], but when it comes to extracting structural/geometric information for 3-D modeling and robot manipulation, the concourse of a user is required to provide hints on the segmentation from mul-tiple views [8]. If a fully automated process is sought, depth information needs to be extracted through stereo [9], struc-tured light [10], or a laser scanner [11]. These techniques have been proven adequate for offline modeling, but they either require special conditions or are too slow to be used in online robot interaction with plants. Recently, time-of-flight (ToF) cameras have been proposed as a good alterna-tive [12] since they provide low-resolution depth images at 25 frames/s. This permits quick acquiring and fusing of the images from different viewpoints [13], which is very useful since one-shot plant data are often partial or ambiguous.

Concerning the robot’s actions, planning and learning algorithms for the manipulation of deformable objects [14]play an important role in this context. Planning must

encompass the motion of the camera as well since plants are prone to occlusions and merging of close leaves; selecting the best next viewpoint may be crucial to disentangle occluded leaves [15] as well as to determine and access suit-able probing points.

Specifically, we address the problem of accurately placing a cutting tool on a leaf to acquire sample discs from plants. Samples drawn at different developmental stages can be used to subsequently analyze their relative growth rates [16].Thus, the emphasis of this article is on sensing-for-actionmethods developed to segment leaves, fit quadratic surfaces to them, determine best candidates for probing, move the cameras to get a closer view, determine a suitable sampling point on the chosen leaf, and finally reach this point with a disc-cutting tool. Intensity-based segmentation is comple-mented with the depth data supplied by a ToF camera to delimit and fit surface patches to the leaves. The ToF camera and the cutting tool are mounted on the robot end-effector (as shown in Figure 1) so that an egocentric coordinate frame is used for all motions.

Overview of the MethodThe probing of a leaf follows a two-stage approach (see Fig-ure 2). Initially, the robot arm is moved to a position from which a general view of the plant is obtained. The depth

Figure 1. (a) The WAM arm used in the experiments holding the ToF sensor, a color camera (data not used), and the cutting tool used to extract samples from the leaves. (b) Typical intensity image. (c) The color-coded 3-D point cloud acquired with a ToF camera [200 # 200 photonic mixer device (PMD) CamCube 3.0].

(a) (c)

(b)

Figure 2. A flow chart of the suggested probing procedure.

Move to Initial Far Position andAcquire Depth/IR Images

Extract and EvaluatePotential Target Leaves

Extract Target Leaf andGrasping Points

Select TargetLeaf

Suitable?No

Yes

Sample Leaf inTwo-Step Path

Move Robot to Close,Frontal View of Target Leaf and

Acquire Depth/IR Images


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







and infrared images acquired with a ToF camera (as described in the next section) are segmented into their composite surfaces as described in the “Depth Segmenta-tion” section. Leaf-model contours are fitted to the extracted segments, the validity of the fit and the graspabil-ity of the leaf are measured, and the segments are ranked (see the “Extraction of Grasping Points” section). A target leaf is selected, and the robot moves the camera to a closer, fronto-parallel view of it. The validity of the target and the graspability are then re-evaluated (see the sections “Con-tour Fitting” and “Graspability”). If the leaf is considered suitable for sampling on the basis of these criteria, then the probing tool is placed onto the leaf following a two-step path (see the “Intermediate Goal Position and Probing Point” section). If the target is not considered suitable for probing, another target leaf (from the general view) is selected, and the procedure is repeated.

3-D Image AcquisitionDepth measurements are acquired with a ToF camera [see Fig-ure 1(a) and (b)]. This type of sensor has the main advantage of providing registered depth and infrared-intensity images of a scene at a high frame rate. The ToF cameras use the well-known ToF principle to compute the depth. The camera emits modulated infrared light to measure the traveling time between the known emitted waves and the ones reflected back by the objects in the scene.

ToF cameras have two main drawbacks: low resolution [e.g., 200 # 200 pixels for a photonic mixer device (PMD) CamCube 3.0 camera] and noisy depth measurements due to systematic and nonsystematic errors [17]. On the one hand, low resolution can be a big problem for large environment applications, but it does not have such a negative impact when the camera is used at close ranges as it is our case. On the other hand, noisy depth measurements due to nonsys-tematic errors are amplified by working in such a short range. Systematic errors are highly reduced by calibration procedures, and nonsystematic errors can be palliated using filtering techniques. Here we apply two filters to remove undesired wrongly estimated point depths and noise: a jump-edge filter and an averaging filter [18].

Depth SegmentationIn this section, we describe an algorithm that segments the sparse and noisy depth data measured by the ToF camera into surface patches to extract task-relevant image regions, i.e., leaves. We assume that plant leaves are usu-ally represented by a single surface in a 3-D space. Although this assumption may not be generally valid, we assume that it holds in most cases. With the many occlu-sions present in grown plants and the variability of leaves in terms of size, orientation, and 3-D shape, the applica-tion of appearance models directly to the image data with the purpose of the leaf segmentation would be extremely challenging, especially since partial shape models might also have to be utilized.

Removing the noise and invalid points in the depth data by using the jump-edge filter provides a sparse depth map. We segment the data by using the infrared-intensity image of the depth sensor as an auxiliary image. Unlike depth, which is measured using the ToF principle, the corresponding infrared-intensity image provides complete (dense) information with little noise. In comparison with color or gray-level images, the infrared intensity images are more amenable to segmentation, since plant-type characteristic color textures are not present here. The segments are then selected and merged on the basis of the available, potentially sparse depth information.

The algorithm proceeds as follows. First, the infrared-inten-sity image is segmented with a standard algorithm at different resolutions. The details can be found in [19]. This is necessary as we do not know beforehand at which resolution good regions will appear. Those segments that fit the depth data best, according to a parametric surface model (see the “Fitting of Quadratic Surface Models” section), are selected, and a new segmentation is constructed. This procedure has been described in detail in [15] and will thus not be repeated here. From this intermediate segmentation and the respective esti-mated parametric surfaces, a graph is built, in which the nodes represent segments, and edges represent the pairwise similarity of the segments’ surfaces, as described in the “Segment Graph” section. Then, to remove remaining over-segmentations pres-ent in the intermediate segmentation, a graph-based merging (clustering) procedure is employed that allows us to handle the nonlocal character of surface properties (see the “Segment Dis-similarity” and “Graph-Based Merging of Segments” sections). An overview of the algorithm is provided in Figure 3.

The method requires currently about s28. to segment an image and to fit surface models using MATLAB and non-optimized code.

Fitting of Quadratic Surface ModelsFor modeling the 3-D surfaces of image regions, we use a quadratic function, which allows us to treat planar, spherical, and cylindrical shapes. Surfaces with more involved curva-tures could also be managed within the same approach but are not required for the application at hand. Moreover, we use quadratic functions that allow computing the depth z explic-itly for the x-y coordinates in the form of , .z f x y= ^ h Thus, surfaces are described by the five parameters , , , ,a b c d and ,ewhere the depth z can be expressed as a function of x and ythrough z ax by cx dy e2 2= + + + + .

For a given segment ,si we perform a minimization of the mean squared distance

( ) ,E N z z1,model ,

2i j j m

j= -/ (1)

of measured depth points z ,j m from the estimated model depth , ,z f x y,modelj i j j= ^ h where f ,modeli is the data-model function and N is the number of measured depth points in the area of segment. The optimization is performed with a Nelder-Mead simplex search algorithm provided in MATLAB.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Segment GraphA nearest-neighbor graph is constructed from the image seg-ments. For each image segment, the boundary points are extracted and the local neighborhood within a radius of 1 px of each point is searched for points belonging to other segments that lie within a predefined absolute depth dis-tance. For computing the depth distance, the fitted depth derived for the respective segment point is used, i.e., ( , ) ( , ) ,f x y f x yi i i j j j- where i and j denote neighboring pix-

els belonging to different segments respectively. Two seg-ments are considered neighbors if the respective boundary points are less than cmd 1D3 = apart. The segments define the nodes V of the segment graph , .V e^ h An edge e exists between two segments if they are neighbors according to the condition given above.

Segment DissimilarityWe define a dissimilarity measure ed between two segments si and s j by estimating how well the surface model of segment si describes the depth data of segment ,s j and vice versa. Let fi be the surface model of segment ,si and ,f j the surface

model of segment .s j Then, we compute the fitting error

/ ( , ) ( , ) ,E n f x y z x y1/2

i j i jp si

= -!

6 @/ (2)

where ,z x y^ h is the measured depth at , ,x y^ h ( , )f x yj is the estimated depth value at , ,x y^ h and ni is the number of points in segment .si The fitting error E /j i is defined accordingly. The surface parameters have been estimated before, hence no surface fitting has to be performed at this step. Then the smaller error of E /i j and E /j i is selected, yielding .ed

Graph-Based Merging of SegmentsThe pairwise dissimilarities between segments are used to sort the graph edges in an order of increasing dissimilarity. For this purpose, we define a label l enumerating the edges in an ascending order. The total number of edges is n . We further define a merging threshold ,dmerge which in our case should be chosen in the range of cm1 5– 2 to be proportional to the expected range of target-fitting errors in the given scenario. The surface models of all graph nodes or segments are stored in a list because they may be updated during the procedure.

The algorithm then proceeds as follows.1) We select the first edge of the ordered list labeled .l 1=2) The two segments linked by the edge labeled l are merged, if

the edge dissimilarity .( )e dl < merged In this case, a new region si j, is created and the respective surface model fi j, is found. The surface models of region si and s j are

replaced by the new surface model .fi j, A flag is set indi-cating whether the surface model of a segment has been updated or not. If ( ) ,e l dmerged F nothing needs to be done.

3) We select the next edge of the ordered list labeled .l l 1= + If one of the segments linked by the respective

edge has been updated previously and thus flagged, the edge dissimilarity between the segments is recomputed using the current surface models.

4) Steps 2 and 3 are repeated until .l n=Working consecutively along the ordered list and updating

the surface models along the way allows us to avoid testing for all possible merging combinations, which otherwise could lead to a combinatorial explosion. This strategy gives prefer-ence to the merges of segments with large similarity. The method is related to Kruskal’s algorithm for finding the mini-mum spanning tree of a graph, except that certain graph edges have to be updated after each merge.

Intensity (PMD)

Imag

e A

cqui

sitio

nIn

frar

ed-I

nten

sity

Seg

men

tatio

nS

egm

ent S

elec

tion

Seg

men

t Gra

phC

onst

ruct

ion

Gra

ph-B

ased

Clu

ster

ing

Iden

tific

atio

n of

Gra

spin

gP

oint

s U

sing

Lea

f Mod

els

Level 0 Level 1

Selected Segments

Segment Graph

Final Segments Fitted Depth +Segment Boundaries

1

0.60.50.40.30.20.10

Segment Validity +Identified Grasp Point

Level 2

Depth (PMD)

Figure 3. The schematic of the leaf-extraction algorithm. ToF data (depth and infrared intensity) is acquired and the infrared-intensity image is segmented at different resolutions (levels 0–2). The surface models are fitted to the segments and those segments along the segmentation hierarchy that fit the depth data best are selected. From the selected segments, a segment graph is constructed and a graph-based segment merging procedure is employed. Final segment contours are fitted to predefined model contours, and grasping points are determined.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Extraction of Grasping PointsWe assume that the above-described procedure delivers seg-ments that correspond to the leaves of the plant. This assump-tion may not always hold, but it is a good enough working hypothesis as we will demonstrate below.

The goal of this article is to identify and model leaves from the ToF data to find suitable grasping points and approach vectors for probing. We use the following strategy. First, a tar-get segment is selected from the processed data obtained from a far (general) view of the plant. Using the surface normal and 3-D position of the target, we move the robot arm with the mounted ToF camera closer to the target and align the viewing direction of the camera with its surface normal. At this close position, a new image is acquired, which we use to confirm or reject our target. If a suitable leaf target is found, a grasping point is identified and an approach to the leaf is planned.

For probing a leaf, two main requirements have to be met by the grasping point so that the task is executable.1) The grasping point should lie within the part of the leaf

that points away from the stem of the plant. Thus, the risk of collisions with the stem and other leaf parts can be reduced. We then want to approach the leaf from the side to maximize the touched leaf area.

2) The grasping point should not be occluded and/or obstructed by other leaves (or objects).To fulfill requirement 1), a leaf-specific contour needs to

be fitted to the leaf-segment boundary to map leaf-specific grasping points along the segment boundary (see the next subsection). The contour-fitting error here gives us a mea-sure of validity of the selected points. The grasping points from 1) are further tested for their graspability using the criteria 2) (see the subsection about the graspability of identi-fied grasping points). Both the contour-fitting error and the graspability measure are important for evaluating whether a planned grasp is executable.

The contour fitting and grasp-point identification require about 2 s for a single segment using MATLAB and nonopti-mized code.

Contour Fitting for Grasping-Point IdentificationWe extract the outer 2-D boundary Ci of segment ,si consist-ing of a set of points , , .x y z" , Before Ci can be compared with the model boundary, we need to rotate the boundary in 3-D to a predefined orientation that aligns its surface nor-

mally with the z-axis. Thus, perspective distortions can be removed at least partially, leading to a point set , .x y r" , We ignore variations in the z-coordinate, since we are only inter-ested in the projection of a leaf boundary onto the x-y plane.

For each plant type, we have extracted the leaf boundary, which is characteristic for the specific plant. We smooth the boundary points with a Gaussian function. The resulting val-ues provide a set of weighted boundary points , , ,x y w m" ,defining our model boundary .Cm

Compared with the model boundary ,Cm which is the characteristic for a specific leaf, Ci might be translated, rotated, or scaled in 2-D. These three transformations pro-vide four parameters, i.e., a translation vector , ,x yt t^ h a rotation angle ,i and scaling factor .a Applying these transformations to Ci leads to a transformed set of points

, , .x y z trans" , We define the distance ,D C Ci m^ h of the transformed boundary to a model boundary for the given transformation parameters by counting the amount of seg-ment-contour points that can be matched with the model contour negatively and the amount of model-contour points, which remain unmatched positively.

We find the parameters of the transformations that provide a best match to the model contour by minimizing the distance

( , )D C Ci m by using a Nelder-Mead simplex search algorithm provided in MATLAB. Once the segment contour is fitted to the model contour, we can identify grasping points. We assume that predefined grasping points are provided together with the leaf contour model, as illustrated in Figure 4. For each model grasping point, we find the point on the segment con-tour that has the smallest distance to the model grasping point. Together with the resulting grasping point x , , ,x y zg g g g=^ hwe also provide the validity measure of the fit.

Graspability of Identified Grasping PointsWe consider a grasping point (which by definition is located on the boundary of the segment) to be graspable if there are no obstructing objects (e.g., other leaves) in its direct vicinity, and if the given boundary is a true leaf boundary, i.e., it is not caused by an occlusion. We define graspability measure g by counting (negatively) the points in a circular area (in 2-D) around the grasping point that belong to another segment and are located within a predefined threshold distance d(here, d 10= cm) from the grasping point, or have a depth value z smaller than zg . The radius of the circular area around the grasping point is chosen equal to .d

Intermediate Goal Position and Probing PointThe probing tool must be placed so that the leaf can slide into the cavity of the tool, which is only 2-cm wide. For this to be successful, the probing tool needs to be aligned with the orien-tation of the leaf. For this purpose, the average surface normal of the leaf is computed. Furthermore, the probing tool is first placed at an intermediate goal position at a certain distance from the grasping point. We compute the intermediate goal position by first defining an approach vector for the grasp according to a x x ,–g g c= where xc is the center point of the

(a) (b) (c)

Figure 4. The model contours for different plants used in our experiments with associated grasping points. (a) Anthurium. (b) Pothos. (c) Dieffembachia. Anthurium was used for experiments 1–3, Pothos for experiment 4, and Dieffembachia for experiment 5. The model contours have been extracted from single selected leaves.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







leaf. The approach vector is normalized and used together with the grasping point to compute the intermediate goal posi-tion x x a ,10g alo g g= + at 10-cm distance from the edge point toward the outside of the leaf.

We further define a probing point at which the tool should be finally placed x x a .2pr bingo g g= - The probing point is located 2 cm from the edge point toward the inside of the leaf.

Experimental SetupThe experimental setup includes a PMD CamCube ToF cam-era and a PointGrey Flea camera rigidly attached to the last link of a Barrett WAM arm (Figure 1). The PointGrey Flea camera is, however, not used in the experiments here. As can be observed, the cameras are displaced from the robot end-effector position to leave room for a cutting tool that we have designed for the given task.

We have opted for a configuration where the cutting tool is outside the field of view of the camera. This implies that, dur-ing the robot motion from the close view of the leaf to the placement of the cutting tool, the leaf is not in the camera’s field of view, and the motion is then executed in open loop. Implicitly we assume that the leaf will not move and that the robot has enough precision along this small motion.

The robot and plant initial relative configuration assures that the plant’s region of interest is reachable by the robot’s cutting tool. In a similar way, plant position is guaranteed to be inside the field of view of the camera’s initial pose. In the close view, the camera is placed in a frontal configuration at 40-cm distance from the localized leaf.

Basic Verification of the MethodThe presented robotic leaf-probing strategy assumes that, for successful sampling of plant leaves, it is advantageous to move first to a closer and fronto-parallel viewing position with respect to the leaf surface. To support this claim, we verify that 1) surface normals of leaves can indeed be accu-rately estimated with the given method, and 2) moving to a closer, fronto-parallel view of a leaf allows better verifica-tion of suitable leaves for probing and thus a better deter-mination of grasping points.

To test assumption 1), we used a planar artificial leaf. For this purpose, the shape of a real leaf (Anthurium) was taken and cut from a carton. The shape was also used as a model leaf for this particular experiment. The artificial leaf was attached to a beam and rotated around its center to attain dif-ferent angles of its surface normal with the viewing direction of the PMD camera. The leaf was rotated in steps of 5°, start-ing at 0°, and a depth and an infrared-intensity image was acquired at each step. Using our method, the leaf was seg-mented and the surface normal was computed by fitting a plane to the 3-D points of the segment. The enclosing angle of the measured surface normal with the z-axis was calculated. Fitting of a line to the data revealed an approximate measure-ment error of about . ,0 7! c which demonstrates that a suffi-ciently accurate estimation of the surface normal can be obtained with the system.

To verify assumption 2), we used the model-leaf contour to calculate the validity of the extracted segments during the previous experiment. The validity measures the correlation between the measured 2-D segment contour and a 2-D model-leaf contour (see the section describing the extraction of grasping points). We observed that the validity decreases as the angle increases, i.e., the further we move away from the fronto-parallel position, the harder leaf recognition becomes because of view-dependent shape distortions and other visi-bility impairments. This also implies that the grasping point cannot be accurately determined past some angle because the model-leaf contour together with the associated grasping point will fit the segment boundary only very poorly. We fur-ther acquired depth and infrared-intensity images from vari-ous viewing angles of a real leaf (the very leaf that had also been used to extract the model-leaf shape in the previous experiment). The computed validities, displayed in Figure 5,are similar to the validities obtained for the artificial leaf. The experiments demonstrate that the method is capable of extracting the target leaf despite varying viewing conditions and without adjusting the parameters. A close view for prob-ing is desirable since it increases the amount of data that can be gathered about a leaf (the resolution), which is immedi-ately evident and does not need to be demonstrated.

Combining the data for the artificial and the real leaf showed that the validity follows roughly a linear relationship. By finding the mean distance of the measured points to the fitted line, we obtained an approximate error measure of the computed validity values of about ±0.1. However, this mea-sure has been computed for leaves that are mostly planar. Bent or curled leaves might show larger errors.

Leaf ProbingAfter verifying the basic assumptions of our approach, we test the method on different plants and for different view-points. Each experiment proceeds in the same way. First, the

Angle (°)

0 10 20 30 40 50 60 70

0.9

0.8

0.7

0.6

0.5

Val

idity

0.4

0.3

0.2

0.1

Figure 5. The validity of the leaf as a function of the measured enclosing angle of the surface normal with the z-axis(camera-viewing direction) for a real leaf (Anthurium). The validity measures the correlation between the measured and transformed 2-D segment contour and a 2-D model-leaf contour.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







plant is examined from a far (general) viewing position of the robot arm. The ToF data is processed and a target leaf is selected. Second, using the target’s pose, a new robot position

is planned and the robot is moved to get a close view of the target. Third, the ToF data from the new view is processed and it is confirmed whether the target is of sufficient validity and graspability

g 10> -^ h. Only then is the reaching movement computed and the grasp exe-cuted. Throughout all the probing experiments, the same set of parameters is used in the algorithms with

cm .d 1merge2= Note that the maxi-

mum validities obtained in these experi-ments are smaller than in the benchmark

experiments since here the model-leaf contour might not perfectly describe the selected leaf because of natural varia-tions in the shape appearance of the leaves.

1

2

3

4

6

5

0.7

0(a)

(b)

(c)

(d)

(e)

Figure 6. The segmentation and target selection results for ToF data acquired from a far viewing position relative to the plant for five different experiments. (a)–(e) correspond to experiments 1–5, respectively. Depth is color coded from red to blue with increasing depth (third and fifth column). The grasping points of selected and labeled target segments are marked with a red star. The validity measures the correlation between the measured and transformed 2-D contour of a segment and a 2-D model-leaf contour. Columns depict (from left): scene, intensity (PMD), depth (PMD), segmentation, fitted surfaces, and segment validity + targets with grasp.

Table 1. Validity, graspability, and enclosing angle a (°) of the surface normal with the camera-viewing direction for the far view and the close view of experiments 1–5 in comparison.

Example vview 1 vview 2 gview 1 gview 2 aview 1 aview 2

1 (Target 1) 0.51 0.48 -31 0 6.9 4.7

1 (Target 2) 0.44 0.46 0 0 5.7 6.5

2 (Target 3) 0.55 0.51 0 0 18.5 7.3

3 (Target 4/7) 0.16 0.5 0 0 15.6 17.0

4 (Target 5) 0.5 0.54 0 -5 1.2 2.6

5 (Target 6) 0.47 0.33 -29 0 26.8 6.0


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







In Figure 6, the results of the analysis of the ToF data for five experiments (1–5) obtained for the far viewing position are shown. In general, depth segmentation delivered sufficiently good results to identify targets of interest. Except for experiment 3 [Figure 6(c)], for which the segmentation failed, targets of suf-ficient validity could always be found. The values of the validity and graspability measure are summarized in Table 1. Targets selected during the experiments are labeled with a unique num-ber in the figures and the table. The computed grasping points are indicated with a star-shaped symbol in the figures.

Based on the selected target, a close view of the target can be planned using its 3-D pose. After moving to the close position, the newly acquired data is analyzed. As can be seen in Figure 7, the segmentation improves in the close view compared with the general view, and in all cases except experiment 3, target leaves can be confirmed indicated by a

sufficiently large validity ( .v 0 32 ). However, in experiment 5, the validity decreased by 0.14, which is nevertheless still in the error margin of the validity computation. The validity measure can be impaired by many factors, e.g., shape differ-ences of the real leaf compared with the model leaf, nonopti-mal solutions encountered by the fitting procedure, the seg-mentation errors distorting the boundaries.

Since the contour models provide just a rough approxi-mation, and the validity estimation is afflicted with some error as well (see the previous section), validities are expected to increase from the far view to the close view only if the change in the viewing angle is large (>30°). Although the graspabilities close to zero indicate that a grasp is executable, the segmentation errors or noise in the data can cause the graspability value to deviate slightly from zero, e.g., experiment 4.

2

3

7

5

6

(a)

(b)

(c)

(d)

(e)

222

3

5555555555

6

7

Figure 7. The segmentation and target selection results for ToF data acquired from a close viewing position relative to the plant for the five different experiments. (a)–(e) correspond to experiments 1–5, respectively. Depth is color coded with increasing depth coded from red to blue (second and fourth column). The grasp points of selected and labeled target segments are marked with a red star. The results for the close view of target 1 are shown in Figure 3. Columns represent (from left): intensity (PMD) close view, depth (PMD), segmentation, fitted surfaces, and segment validity.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The close view in experiment 3 provides more informa-tion on the plant than the far view. Now several leaves can be distinguished, reflected in the larger validity of the segments. As a consequence, a better target (labeled 7) with a higher validity can be selected for the grasp.

Furthermore, moving from a far view to a close view allows us to collect more data about a segment and to bring the camera to a fronto-parallel position with respect to the leaf. In Table 1, the enclosing angle of the surface normals with the camera-viewing angles of the leaf for the far and the close view are reported, showing that surface normals have been re-estimated after going to the new view and are suffi-ciently aligned with the z-axis for the close view. In experi-ment 3, however, the viewing direction of the camera could not be aligned well with the surface normal of the leaf in the close view, because two leaves were merged in the far view, and thus no surface normal of a singular leaf could be iso-lated at the beginning.

The target segments and their respective grasping points are used to compute the approach for probing the leaf. In Figure 8, the 3-D point cloud of target 2 is presented together with the grasp point (hexagram), the center point of the tar-get (circle), the probing point (diamond), the intermediate goal position (square), the surface normal (black line), and the approach vector connecting all these points (green line). Using this information, the intermediate goal position of the robot can be calculated.

The grasps were then successfully executed by first going to the intermediate goal position and then advancing to the probing position. Images of the probing for the experiments are presented in Figure 9(a)–(e). Once the cutting tool is cor-rectly placed, a small sample of the leaf can be taken by cut-ting out a small disc. In Figure 9(f), an image of a leaf after sampling is shown.

The accurate placement of the probing tool indicates that surfaces have been correctly estimated by the approach. The successful execution can be partly attributed to precise leaf estimation, i.e., surface normal and grasping points, which

could be obtained using the data acquired from the close-view position.

Movies of the experiments can be found at http://www.iri.upc.edu/people/galenya/pub/LeafProbing.

ConclusionWe presented a method for modeling, monitoring, and sam-pling plant leaves using infrared-intensity images and depth maps acquired with a PMD camera. Since quadratic surface models are used to guide the segmentation of the infrared-intensity image, either sparse or noisy depth data can be used. This kind of data often poses a problem to approaches working in depth space directly. In such a situation, segments are ranked, and a closer view of the candidate that most likely represents a suitable leaf is taken. Thus, ambiguities can be cleared up. For example, in the experiments, two leaves that were initially merged into a single segment could be separated and modeled individually in the close view. The grasping points could be extracted with high accuracy, and disc samples could be cut successfully.

The problem of leaf segmentation has been addressed before by Quan et al. [8], who in 2006 proposed an image-based plant modeling system on the basis of structure from motion, which requires user interaction in the segmentation procedure to delineate some leaves. In another related work, leaves were segmented from combined color images and ste-reo depth and subsequently classified using the normalized centroid contour distance [7]. Unlike these approaches, we

z (cm)

y (cm)

x (cm)-8 -4 0128

4

424038

0

-4-8

4 8

Figure 8. The target point cloud and approach points. The 3-D points of the selected target segment 2 of experiment 1 [seeFigures 6 and 7(a)] are shown together with the associated center point (circle), probing point (diamond), grasping point (hexagram), the intermediate goal position (square), the surface normal (black line), and the approach vector (green line), which intersects all these points. Distances are given in centimeters.

(a) (b)

(c) (d)

(e) (f)

Figure 9. (a)–(e) Color images documenting the successful execution of grasps. The probing tool could be accurately placed on the leaf, and a disc-shaped piece of the leaf could be cut. (f) A leaf after a sample has been taken with the cutting tool.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________________________

http://www.qmags.com/clickthrough.asp?url=http://www.iri.upc.edu/people/galenya/pub/LeafProbing&id=18410&adid=P58E1






extract leaves from ToF data and infrared-intensity images. The segmentation is fully automatic and is based on a novel depth-segmentation algorithm, which can be applied to sparse or noisy depth data and can cope with curved surfaces. Another difference is that leaf models are fitted explicitly, which allows us to localize grasping points.

The proposed system for automated plant probing is related to vision-based robotic systems for fruit and vegetable picking that have been proposed in the past for the automation of harvesting tasks [6]. Often these systems first process and segment the data to identify and represent the target. Based on this representation, a robot action (e.g., cutting or grasping) is executed. The image-processing task is often eased by fixing the environment in a specific manner. For example, in a fruit-detachment system developed by Feng et al. [20] in 2008, strawberries were grown on a uniformly colored surface to simplify image segmentation. In our system, the environment is less constrained, and the proposed computer-vision system is thus more complex. Furthermore, a new robotic application, i.e., the automatic sampling of leaves with a specific cutting tool, was introduced and explored. To the authors’ knowledge, this is the first time that an active vision approach using ToF depth has been applied to robotized plant measuring.

The method is based on several assumptions: 1) the boundaries of leaves are visible in the infrared-intensity image, 2) the leaf surfaces can be modeled by a basic qua-dratic function, 3) leaves of a specific plant type can be described by a common 2-D contour, 4) leaves are large enough to be analyzed with a ToF camera, and 5) the leaves are static during probing. These assumptions may be violated under certain conditions, but nevertheless we expect the method to be applicable to many different types of plants, given a controlled environment.

In conclusion, we tackled a quite complex task that required the extraction of task-relevant plant parameters from plant images using a multistage algorithm as well as the diffi-cult problem of the actual execution of the robot motion toward the plant. The automation of plant probing has a potentially wide range of applications both in the agricultural industry, in which certain (currently manual) tasks must be repeatedly executed for multiple plants, and in botanic experi-mentation, e.g., for phenotyping, in which leaf sample discs are commonly used to analyze plant development to deter-mine the genetic factors that control growth.

AcknowledgmentsThis research was partially funded by EU GARNICS project FP7-247947, project PAU+ (DPI2011-27510), and Grup con-solidat SGR155. B. Dellen acknowledges support from the Spanish Ministry of Science and Innovation through a Ramon y Cajal program. S. Foix was supported by a Ph.D. fel-lowship from CSIC’s JAE program.

References[1] A. Kolb, E. Barth, and R. Koch, “ToF-sensors: New dimensions for realism and interactivity,” in Proc. IEEE CVPR Workshop, 2008, vols. 1–3, pp. 1518–1523.

[2] A. Nealen, M. Muller, R. Keiser, E. Boxerman, and M. Carlson, “Physically based deformable models in computer graphics,” Comput. Graphics Forum,vol. 25, no. 4, pp. 809–836, 2006.[3] R. B. Rusu, A. Holzbach, R. Diankov, G. Bradski, and M. Beetz, “Perception for mobile manipulation and grasping using active stereo,” in Proc. 9th IEEE-RAS Int. Conf. Humanoid Robots, 2009, pp. 632–638.[4] T. Fourcaud, X. Zhang, A. Stokes, H. Lambers, and C. KÖner, “Plant growth modelling and applications: The increasing importance of plant archi-tecture in growth models,” in Proc. Ann. Botany, 2008, pp. 1053–1063.[5] E. J. Van Henten, B. A. J. Van Tuijl, G.-J. Hoogakker, M. J. Van Der Weerd, J.Hemming, J. G. Kornet, and J. Bontsema, “An autonomous robot for de-leafing cucumber plants grown in a high-wire cultivation system,” Biosyst. Eng., vol. 94, no. 3, pp. 317–323, 2006.[6] T. Grift, Q. Zhang, N. Kondo, and K. C. Ting, “A review of automation and robotics for the bio-industry,” J. Biomech. Eng., vol. 1, no. 1, pp. 37–54, 2008.[7] C.-H. Teng, Y.-T. Kuo, and Y.-S. Chen, “Leaf segmentation, classification, and three-dimensional recovery from a few images with close viewpoints,” Optical Eng., vol. 50, no. 3, 037003, pp. 1–12, 2011.[8] L. Quan, P. Tan, G. Zeng, L. Yuan, J. Wang, and S. B. Kang, “Image based plant modelling,” ACM Siggraph, vol. 25, no. 3, pp. 599–604, 2006.[9] Y. Song, R. Wilson, R. Edmondson, and N. Parsons, “Surface modelling of plants from stereo images,” in Proc. 6th IEEE Int. Conf. 3D Digital Imaging Modelling, 2007, pp. 312–319.[10] G. Taylor and L. Kleeman, “Robust range data segmentation using geo-metric primitives for robotic applications,” in Proc. 5th IASTED Int. Conf. Sig-nal Image Processing, 2003, pp. 1–6.[11] B. I. Loch, J. A. Belward, and J. S. Hanan, “Application of surface fitting techniques for the representation of leaf surfaces,” in Proc. MODSIM Int. Congr. Modelling Simulation, 2005, pp. 1272–1278.[12] R. Klose, J. Penlington, and A. Ruckelshausen, “Usability study of 3D time-of-flight cameras for automatic plant phenotyping,” in Proc. Workshop Computer Image Analysis Agriculture, 2009, pp. 93–105.[13] S. Foix, G. Alenyà, J. Andrade-Cetto, and C. Torras, “Object modeling using a ToF camera under an uncertainty reduction approach,” in Proc. IEEE Int. Conf. Robotics Automation, 2010, pp. 1306–1312.[14] F. F. Khalil and P. Payeur, “Dexterous robotic manipulation of deformable objects with multi-sensory feedback-a review,” in Robot Manipulators Trends and Development, A. Jimenez and B. M. A. Hadithi, Eds. 2010, pp. 587–619.[15] G. Alenyà, B. Dellen, and C. Torras, “3D modelling of leaves from color and ToF data for robotized plant measuring,” in Proc. IEEE Int. Conf. Robotics Automation, 2011, pp. 3408–3414.[16] B. Biskup, H. Scharr, A. Fischbach, A. Wiese-Klinkenberg, U. Schurr, and A. Walter, “Diel growth cycle of isolated leaf discs analyzed with a novel, high-throughput three-dimensional imaging method is identical to that of intact leaves,” Plant Physiol., vol. 149, no. 3, pp. 1452–1461, Mar. 2009.[17] S. Foix, G. Alenyà, and C. Torras, “Lock-in time-of-flight cameras: A sur-vey,” IEEE Sensors J., vol. 11, no. 9, pp. 1917–1926, 2011.[18] S. Fuchs and S. May, “Calibration and registration for precise surface reconstruction with time of f light cameras,” Int. J. Int. Syst. Tech. App., vol. 5, nos. 3–4, pp. 274–284, 2008.[19] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient graph-based image segmentation,” Int. J. Comput. Vision, vol. 59, no. 2, pp. 167–181, 2004.[20] G. Feng, C. Qizin, and M. Masateru, “Fruit detachment and classification method for strawberry harvesting robot,” Int. J. Adv. Robot. Syst., vol. 5, no. 1,pp. 41–48, 2008.

Guillem Alenyà, Institut de Robotica i Informatica Industrial (CSIC-UPC), Barcelona, Spain. E-mail: [email protected].

Babette Dellen, Institut de Robotica i Informatica Industrial (CSIC-UPC), Barcelona, Spain. E-mail: [email protected].

Sergi Foix, Institut de Robotica i Informatica Industrial (CSIC-UPC), Barcelona, Spain. E-mail: [email protected].

Carme Torras, Institut de Robotica i Informatica Industrial (CSIC-UPC), Barcelona, Spain. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


___________

___________

_____________

_____________











n this article, we present a robot capable of autonomously traversing and manipulating a three-dimensional (3-D) truss structure. The robot can approach and traverse multiple structural joints using a combination of translational and rotational motions. A key factor in allowing reliable motion and engagement is the use of

specially designed structural building blocks comprised of bidirectional geared rods. A set of traversal plans, each comprised of basic motion primitives, were analyzed for speed, robustness, and repeatability. Paths covering eight joints are demonstrated, as well as automatic element assembly and disassembly. We suggest that the robot architecture and truss module design, such as the one presented here, could open the door to robotically assembled,

maintained, and reconfigured structures that would ordinarily be difficult, risky, or time consuming for humans to construct.

Design of RobotStructure-climbing robots have traditionally been devel-oped to perform tasks currently carried out by humans, which range from structural inspections to cleaning and maintenance. Our goal in this article is to explore both robot and structure design to expand the range of tasks which can be successfully completed autonomously. Spe-cifically, we are interested in the design of a robot capable of modifying a structure by taking it apart and rebuilding it into a different shapes. Such structure-reconfiguring robots could have a profound impact on construction processes, especially activities involving frame construc-tion. In the longer term, the ability to autonomously


By Franz Nigl, Shuguang Li, Jeremy E. Blum, and Hod Lipson


Autonomous Truss Reconfiguration and Manipulation

© IS

TOC

KP

HO

TO.C

OM

/RA

DIS

T77

7

I


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Table 1. Summary of climbing robots.

Author (Robot) Connecting Mechanism (Structure Type) Description and Purpose

Aracil et al., 2006 [5] Mechanical gripper Parallel-climbing robot based on Stewart–Gough’s platform for climbing on palm trees and within complex structures.

Balaguer et al., 2002 (Roma 1) [6]

Mechanical gripper Large, untethered inspection robot for structures. Robot mass is 70 kg.

Balaguer et al., 2002(Roma 2) [6]

Suction cups (smooth surfaces) Inspection robot for moving within structures.

Chatzakos et al.,2006 [7]

Mechanical pipe-clamping mechanism Tethered, omnidirectional pipe inspection robot capable of traversing pipe bends and branches.

Chen and Yeo, 2003 [8]

Suction cups Tethered walking robot capable of traversing on flat surfaces.

Daltorio et al.,2009 [9]

Mechanical and adhesive mechanism Small robot that demonstrates climbing with different mechanic and adhesive connection mechanisms that include hooks, spines, adhesive tapes, and Velcro.

Elliot et al., 2007(City-Climber) [10]

Suction mechanism Untethered wall-climbing robot.

Fu et al., 2008 [11] Suction mechanism Wheel-leg hybrid robot capable of moving on horizontal and vertical surfaces.

Goldman, 2009(HyDRAS) [12]

Robot clamps around structure. Snake like climbing robot.

Hillenbrand et al.,2008 (Cromsci) [13]

Suction mechanism Inspection robot for concrete walls.

Hjelle, 2009 [27] Bidirectional gearing system Three-dimensional (3-D) truss construction robot.

Kalra et al.,2006 [14]

Permanent magnet mechanism Oil tank inspection robot.

Kennedy et al., 2006(Lemur IIb) [15]

Mechanical gripping/holding mechanism Free-climbing robot.

Kotay and Rus, 1996 [16]

Magnetic mechanism Tethered inchworm robot for climbing on 3-D structures.

Krosuri and Minor, 2005 [17]

Suction or magnetic mechanism Tethered biped robot for climbing and walking using a hybrid hip joint.

Minor and Mukherjee, 2003 [18]

Suction cups Small, tethered biped robot.

Sattar et al.,2009 [19]

Mechanical mechanism Ring-climbing robot for inspection of wind turbines. Ring-type robot is assembled around wind turbine pole.

Skaff et al., 2001 (Skyworker) [1]

Mechanical gripper Climbing robot with the ability to add and remove individual beams under simulated zero gravity.

Spenko et al., 2008 (RiSE) [20]

Interlocking and bonding mechanism Biologically inspired hexapod untethered robot that uses locking and bonding mechanism to climb walls and trees.

Sun et al., 2004 [21] Suction cups (glass surface) Cleaning robot for glass walls of high-rise buildings.

Tâche et al., 2009(MagneBike) [22]

Magnetic mechanism (ferrous pipes) Two-wheel bike like tethered robot for inspection of large ferromagnetic pipes with complex-shaped geometries.

Tavakoli et al.,2005 [23]

Mechanical gripper Parallel/serial hybrid pole-climbing robot.

Tavakoli et al., 2008(3-DCLIMBER) [24]

Mechanical gripper Tethered climbing robot for inspection of 3-D structures.

Terada et al.,2008 [29](AMAS) [29]

Mechanical gripper Robot that can build and climb on a structure created of passive cube elements.

White et al.,2005 [25]

Suction mechanism Tethered manufacturing and inspection robot.

Yun et al., 2008(Shady3-D) [2]

Mechanical gripper Structure-climbing robot. Three motive degree of freedom (DoF) robot that can combine with a second robot to create a six-DoF robot.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







repair a damaged structure or autonomously adapt an existing structure to a new function can have applications ranging from disaster recovery to space exploration.

One of the key challenges in designing a structure-recon-figuring robot is that most current structural building blocks are designed for manipulation by humans, not by machines. Structural joints require complex assembly, and truss

elements are cumbersome to manipulate. The development of standards for joint connections and elements has been crucial in the progress of modular assembly in the construction industry. Furthermore, the lack of robot-friendly joints and elements has significantly limited the deployment of robots in the construction field.

We explore the codesign of both a robot and structural components for the explicit purpose of robotic structure reconfiguration. Two different robots are described. Robot version one (R1) is used to demonstrate motion within a test structure. By executing a series of climbing motions, R1 can maneuver between the horizontal and vertical planes of a structure and traverse multiple nodes in one run. The second version of the robot (R2) is capable of demonstrating disas-sembly of an entire vertical structural plane and assembly of individual truss components. The two robots represent two generations of development in our project. A picture of the concept as well as robot R2 during disassembly can be seen in Figure 1.

Background and Previous WorkNumerous designs exist that demonstrate robots climbing on walls, traversing poles, and navigating truss-like or tubular structures. Only a few present the ability to manipulate these structures or demonstrate performance statistics about traversing multiple joints—a key perfor-mance metric.

An overview of the different climbing robots is provided in Table 1. Attachment mechanisms used include suction cups, magnetic mechanisms, gripping mechanisms, and adhesive mechanisms. From this group, only Skyworker [1]demonstrated the ability to add a beam to a structure. Shady 3-D [2], [26], [28] demonstrated the ability to create a higher degrees of freedom (DoF) robot using two Shady 3-D robots connected with a passive module. Several struc-tural designs were simulated that could be built with pas-sive elements and Shady 3-D robots. Dogget [3] designed a

Female Bidirectional Gear Male Bidirectional Gear Chairlike Structure

Assembled JointRigid Connector

(b)(a)

Figure 2. Truss structure: (a) design of bidirectional gear rod and (b) fully assembled chairlike structure using rigid connectors.

(a)

(b)

Figure 1. Truss-reconfiguring robot: (a) concept and (b) implementation.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







robot capable of building a 3-D structure composed of 102 truss elements, but the robot is stationary and cannot climb to any point in the structure for inspection or repair. Gallo-way et al. [4] built a similar system that can create 3-D structures layer by layer using a robot fixed to the floor. Automatic modular assembly system (AMAS) [29] includes an assembler robot and passive cube-shaped building blocks. It showed the automatic construction capability for 3-D structures. The robot presented in this article is a fur-ther development of a design shown by Hjelle [27]. This robot exhibited two identical halves connected via a hinge joint. Each side had a translational and a rotational mecha-nism, similar to the robot presented in this article. The robot did not exhibit any feedback and was open-loop con-trolled. The robot engaged onto a 1/8-in pinion wire with 16 threads per inch. Testing of the original design showed the need for improvements of the hardware and electronics. The pinion wire gearing/threading was not adequate and needed bigger teeth to allow better engagement by the robot mechanism. Additionally, it was determined that position feedback from the motors would be required if sig-nificantly improved control was to be achieved. All of these issues are addressed and resolved using the robots demon-strated in this article.

Structural Truss DesignThe structural components have to fulfill three primary crite-ria to ensure a reliable interaction with the robot. First, the parts have to be strong enough to support the robot but light enough to be handled by the robot. Second, the robot has to lock and unlock the connectors, so the entire structure may be reconfigured. Lastly, the robot has to reliably engage, rotate, and translate on an individual rod.

Initial designs utilized cylindrical rods which the robot engaged via rubber wheels. Using this method of attachment, translation and rotation of the robot along and around the rod resulted in slippage. To avoid the problem of slippage and to increase the power transmitted from the robot to the rod, bidirectional gearing was developed for the rods and the robot [Figure 2(a)].

Rod DesignThese novel bidirectional geared rods have gearing in the longitudinal direction as well as in the rotational direction, allowing for increased power transmission from the robot to the rod regardless of the plane of travel. Uniquely, these bidirectional gears allow motion in one direction while inhibiting motion in the orthogonal direction, thus arrest-ing slippage. This enables the robot to remain at the unsta-ble position on top of the rod while successfully translating along the rod.

The bidirectional gearing system consists of a pair of gears: a female bidirectional gear and a male bidirectional gear. The female bidirectional gearing is used on the rods, whereas the robot uses the male bidirectional gearing with its servos to effectively engage the structure.

To create the female bidirectional gears, a contour of a spur gear rack was revolved around a rod axis and was merged with a spur gear thread in the orthogonal direction. The different design stages of the female bidirectionally geared rod can be seen in Figure 2(a). To create the male bidirectional gear, the outside contour of a spur gear rack was revolved around and subtracted from a spur gear. Figure 2(a) shows the CAD drawings of the steps on sample gears and pictures of the 3-D printed actual bidirectional gears.

Connector DesignTo join the rods together at their vertices, two different types of connectors were designed: a fixed connector and a robot-lockable connector. A fixed connector, for traversal by robot R1, was designed to securely connect structural elements together, which could not be manipulated by the robot. In this structural iteration, the angles between the rods are nearly orthogonal, which required less flexibility from the robot to

(a) (b)

(d)

(e)

(c)

Figure 3. Assembly of connector: (a) rod approaching connector, (b) rod in unlocked state on connector, (c) rod twisting into locking position, (d) rod in locked position, and (e) photo of 3-D printed connector node.

Figure 4. Rod with female part of lockable connector.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







compensate for non-90° angles, and resulted in a greater chance of success when traversing over nodes. In the second design iteration, the connectors were modified to allow them to be locked and unlocked by robot R2. These robot-configu-rable joints were needed to allow the robot to independently reconfigure the structure. However, they resulted in more flexible connection points at the nodes of the truss structure, which necessitated a robot with better compensation for non-orthogonal joints between rods. Also, the current design allows the robot to open and close the connectors from only one direction and hence disassemble or assemble a rod from a specific side only.1) Rigid Connectors—Iteration 1 (Robot Traversal Only): In the

rigid connection scheme, individual rods are connected to the center nodes by simple extensions. The rigid connec-

tors can be seen in Figure 2(b). Truss elements were print-ed in pieces, as an entire truss could not be printed because of the size restrictions of the 3-D printer. A node with four rods connected to it can be seen in Figure 2(b).

2) Lockable Connectors—Iteration 2 (Robot Manipulation):Robot R2 was designed with the intention that it would reassemble a truss structure; connectors were redesigned to enable this action. The lockable connectors can be locked and unlocked by the robot and hence they allow the disconnection and removal of individual trusses from the structure. The lockable connectors consist of a male connector part on the node (Figure 3) and a female part on the truss element (Figure 4). These connectors are closed or opened by turning the cylindrical locking element 180°, as illustrated in Figure 3. The rods them-selves were stiffened using a carbon fiber rod as can be seen in Figure 4.

Robot DesignTwo different versions of the robot were built. Robot iteration 1 (R1) was designed to demonstrate traversal in the fixed con-nector structure. As lockable connectors were introduced in the structural design, deficiencies observed with R1 required a design upgrade to robot iteration 2 (R2). This robot has stronger servos, uses sensors, and includes an upgraded con-troller and software.

For both robots, the body consists of 3- and 6-mm-thick laser-cut acrylic, rapid-prototyped plastic printed by a Stratasys Dimension 3-D printer, and gears printed by an Objet Eden 260 V 3-D printer. Nine servos on each robot, controlled by an Atmel-based microcontroller board and accompanying software, provide the actuation. Both versions of the robot use servo and control units from Robotis Inc.

Traversal Robot—Iteration 1 (R1)R1 consists of two mechanically identical halves connected via a hinge. Each side is comprised of a translational and a

rotational mechanism and utilizes a total of four actuators. The translational mecha-nism carries out the longitudinal robot movements along the rod, whereas the rotational mechanism drives angular motions around the rod. R1 was built to demonstrate traversal in a 3-D structure. Its main dimensions can be seen in Table 2.

A single actuator is used to provide hinge motion between the two identical robot halves. It is directly connected to each side without any gearing.

The translational mechanism on each side consists of two servo actuators. One servo utilizes a cam to engage the transla-tional system, and the other servo propels the robot along the rod using a 12-tooth bidirectional male gear engaged on the rod.

Table 2. Comparison of two robots.

Parameter (Component) Robot R1 Robot R2

Height 89 mm 119 mm

Length 318 mm 318 mm

Width 178 mm 190 mm

Mass 1633 g 1740 g

Translational cam servo (x2)

AX-12+ AX-12+

Translational servo (x2) AX-12+ AX-12+

Rotational cam servo (x2) AX-12+ AX-12+

Rotational servo (x2) AX-12+ RX-28

Hinge servo (x1) AX-12+ RX-28

Controller CM-5 CM-2+

Battery location On side of robot

At bottom of robot

Programming Graphical C-Code

Position feedback Rotational angle of servos

Rotational angle of servos, reflectivity sensor feedback

Side View

Top View

Side View

Top ViewTranslational Mechanism

Hinge

Rotational Mechanism

Figure 5. Robot motion mechanisms.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The self-locking engagement system is powered down follow-ing successful engagement to preserve battery power. The translational mechanism is displayed in Figure 5.

Similar to the translational system, the rotational mecha-nism uses an actuated cam to engage the rotational motor. The cam servo locks automatically and is powered down when the rotational assembly has been engaged successfully. In contrast to the translational mechanism, the rotational actuator gear is not directly engaged with the rod, because it is not sufficiently large enough to transfer the necessary amount of torque needed for the robot to perform all required motions. Therefore, a 12:22 gear ratio is utilized to increase the torque delivered to the rod. The rotational mechanism is displayed in Figure 5. It is comprised of a 12-tooth female bidirectional gear and a 22-tooth male bidirectional gear. The female gear is connected to the output shaft of the actuator and the male gear is designed to engage with the structure.

R1 exclusively uses AX-12+ (1.2-Nm torque) servos in combination with a CM-5 control module. No feedback through external sensors is used for this robot. The servos communicate with the control module via a TTL signal and can be dynamically changed between continuous rotational mode and angular servo mode. Limits for torque, speed, angular position, and temperature values are all stored in onboard memory in each servo; these values can be set or read by the program running on the control module. Power for the system is provided entirely by a 12-V rechargeable lithium ion battery, allowing for untethered operation.

Robot version R1 was programmed using the Behavior Control Program graphical programming language. The pro-gram was compiled into a hex file and downloaded onto the control module where it was executed. Control of R1 is achieved entirely by setting motors to prescribed angular rota-tions in the program.

Manipulation Robot—Iteration 2 (R2)Robot R2 is an improved version of R1 that is capable of manipulating the lockable connectors and can therefore per-form more complex assembly and disassembly actions. Imbal-ance caused by the battery, weak servos, and simplistic programming make manipulating the lockable connectors using R1 highly unreliable. These problems were addressed and resolved in R2.

The translational motion is the same as in robot R1. The design proved satisfactory for the translation of the robot in the structure and hence no changes had to be made.

Tests with R1 showed that the servos for the rotational mechanism needed to be upgraded to improve reliability of the 180° rotation motion. The rotational motors were switched out in favor of RX-28 (2.7-Nm torque) servos, which use the RS485 communication protocol in contrast to the half-duplex asynchronous serial communication used with the AX-12+ servos.

Preliminary disassembly tests showed that the hinge servo in robot version R1 needed to be replaced with a stronger

motor to ensure that the robot could properly and reliably react to deviations in the orthogonal angle that was expected at joints. Therefore the hinge servo was replaced with a RX-28 servo as well.

Unlike in R1, the 12-V battery was mounted to the under-side of the robot [Figure 6(b)] to effectively eliminate the weight distribution problem faced by R1.

Upgrading the CM-5 to the CM-2+ controller was neces-sary both to communicate on the RS485 bus used by the stronger servos and to communicate with external sensors. R2 was outfitted with two Fairchild Semiconductor QRD1114 reflective object sensors and accompanying cir-cuitry to send analog sensor information to the controller via an intermediary I/O board. By detecting white acrylic paint markings on the rods (Figure 4) using these sensors, it was possible for the robot to determine its relative position and to proceed accordingly. Software filtering and an adjustable hardware mounting bracket allow these sensors to be cali-brated as needed.

Robot R2’s microcontroller was programmed entirely using the C language. This resulted in a far more advanced control scheme, and the ability to react based on sensor inputs. Isolated motions were preprogrammed (i.e., advanc-ing forward, releasing a truss, etc.), but the timing of these actions was determined by filtered information from the reflectivity sensors. Consequently, R2 could exhibit autono-mous translational movement, whereas R1 could not.

Traversal Motions (Robot R1)To achieve relocation within the structure, the robot needed to perform a series of basic motions. We identified three basic motions that needed to be performed by the robot to

(b)

(a)

Figure 6. Two versions of the robot: (a) Robot R1 and (b) Robot R2.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







reach any point in a cubed truss structure. We used a chair-like test bed, as this structure exhibits a vertical and horizon-tal plane. This permitted us to demonstrate traversal in both planes, as well as transitions between the planes.

Basic MotionsThree basic motions were identified, which in conjunction with one another, allow the robot to reach any point within the structure: translational motion, 90° vertical rotation, and 180° horizontal rotation.1) Translational Motion: To advance in a 3-D structure,

the robot has to be capable of moving along a truss. The robot’s starting position has both robot halves connected to rods with the translational mechanism engaged. The robot shown in Table 3 starts attached to the bottom of a horizontal rod. The robot could also be attached to the top of the horizontal rod as an alternative starting point (the path would be the same, but inverted). The robot then performs the steps as shown and described in Table 3.

This leaves the robot in a position having both transla-tional mechanisms engaged at the center of the rod. The same steps would be performed, but in reverse order, to advance to the other end of the rod. The same motion and steps can be performed on a vertical rod with the robot moving either up or down.

2) 90° Vertical Rotation: This motion lets the robot move between rods in the horizontal plane with the aid of a vertical rod. The initial position is the same as for the translation motion. Both translation mechanisms are engaged on two perpendicular rods. As shown in Table 3, the robot is located at the bottom of the hori-zontal rod. The robot could also start out by being attached above the horizontal rod. The robot then per-forms the steps as shown and described in Table 3.

3) 180° Horizontal Rotation: This motion lets the robot move from the underside of a horizontal rod to the top and vice versa. The starting position engages both rota-tional mechanisms of the robot on the middle of a rod. The robot could either be above or below the horizon-tal rod. When starting below the rod, the robot per-forms the steps as shown and described in Table 3.

This results in a final position that is nearly identical to the starting point for the translational motion. The slight difference is that only one translational mechanism is engaged. Therefore, when continuing with the transla-tional motion, the command to release the translation mechanism will not be executed.

Test BedTo test the robot motion within the structure and to evaluate the robot’s performance with respect to the different basic

Table 3. Basic motions of Robot R1.

Basic Motions Steps

Translational motion 1) Disengage one translational mechanism.2) Decrease the hinge angle.3) Activate second translational mechanism and move robot

to the center of the rod.4) Operate the hinge to align the second translational

mechanism to the rod and engage it.

90° vertical rotation 1) Disengage the translational mechanism on the horizontal rod.

2) Reduce the hinge angle slightly.3) Move robot on vertical rod away from node.4) Engage the rotational mechanism on the vertical rod, disen-

gage the translational mechanism and rotate the robot 90° in the desired direction.

5) Engage the translational mechanism on the vertical rod and disengage the rotational mechanism. Move the robot to the horizontal rod.

6) Move the hinge so that the two robot halves are perpendicu-lar. Engage the translational mechanism on the horizontal rod.

180° horizontal rotation 1) Engage the rotational mechanism and disengage the transla-tional mechanism on each robot half.

2) Rotate the robot 180° by using both rotational mechanisms.3) Engage the translational mechanism and disengage the

rotational mechanism on each robot half.

(1) (2) (3) (4)(1) (2) (3) (4)

(1) (2) (3)

(5) (6)

(4)

(1) (2) (3)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







motions, R1 was programmed to navigate throughout a chair-like structure.

This structure comprises a vertical and a horizontal square plane, which is the minimal needed test bed for the robot. Any bigger sized structure would just consist of additional vertical and horizontal planes. This structure also allows for the testing of the robot’s transition between the two planes. The steps performed by the robot can be seen in Figure 7 and are described below.Step 1) Start out on the front, lower side of the horizontal

rod and travel to the right.Step 2) Move to the right lower horizontal rod and travel

to the end of the right lower horizontal rod.

Step 3) Move to the lower back rod.Step 4) Rotate to the top of the lower back rod.Step 5) Move to the vertical left rod.Step 6) Connect to the left vertical rod and move upwards

to the horizontal top rod.Step 7) Traverse at the bottom of the top vertical rod.Step 8) Move down the right vertical rod.Step 9) Move to the top side of the back horizontal rod.Step 10) Rotate to the bottom side of the back horizontal

rod.Step 11) Move to the left side of the back horizontal rod.Step 12) Move to the left lower rod before moving to the

front lower bar (the original position).

Table 4. Basic motions of Robot R2.

Basic Motions Steps

Translational motion 1) Disengage one translational mechanism. 2) Activate second translational mechanism and move

the robot until the reflectivity sensors detect the stop marking.

3) Operate the hinge to align the second translational mechanism to the rod and engage its translational mechanism.

Disassembly motion 1) Open the hinge slightly and engage the second transla-tional mechanism, adjust the hinge angle to 90°. (This step is critical for ensuring that there is no discrepancy between the angle of the male and female gearings.)

2) Disengage the translational mechanism halfway (so as to hold the rod in place without preventing it from rotat-ing) and engage the rotational mechanism.

3) Disengage the translational mechanism completely and operate the rotational mechanism for 180°, unlocking the connector. Disengage the rotational mechanism to release the rod.

Vertical assembly motion 1) Align rod with connector.2) Operate the rotational mechanism for 180° to lock the

connector. Disengage the rotational mechanism and engage the translational mechanism. (If the rotational mechanism was not turned 180°, the engagement of the translational mechanism will correct misalignments up to 15°.)

Disassembly, 90° rotation, and reassembly of truss 1) Perform disassembly motion (1)–(3) as described above on horizontal rod without performing the last step of releasing the rod.

2) Decrease hinge angle slightly and move robot away from node.

3) Engage rotational mechanism on vertical rod and disen-gage translational mechanism.

4) Rotate robot 90°.5) Engage translational mechanism on vertical rod and

disengage rotational mechanism. 6) Set hinge angle to 90° and translate robot to node using

feedback from reflectivity sensors.7) Operate the rotational mechanism for 180°, locking the

connector. Disengage the rotational mechanism and engage the translational mechanism. (If the rotational mechanism was not turned 180°, the engagement of the translational mechanism will correct misalignments up to 15°.)

(1) (2) (3)

(1) (2) (3)

(1) (2)

(1) (2) (3)

(5) (6) (7)

(4)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Manipulation Motions (Robot R2)R2 was designed to traverse a structure more robustly than R1 while adding the ability to manipulate the lockable truss con-nections. The robot demonstrated disassembly of a vertical square and disassembly and reassembly of a vertical and hori-zontal rod. These actions are shown in the demo video that accompanies this article.

Basic Motions The three basic motions tested were translational motion, dis-assembly motion, and assembly motion. There are slight dif-ferences in the translational motions performed by robots R1 and R2 because of the use of stronger servos and the presence of sensors in robot R2.1) Translational Motion: The robot starting position has both

robot halves connected to rods with the translational mechanism engaged. The robot is attached below a hori-zontal rod. The robot could also be attached on top of the horizontal rod as an alternative starting point. The robot then performs the steps, as shown and described in Table 4.

The robot completes the motion with both translational mechanisms engaged at the center of the rod. The same steps would be performed, but in reverse order, to advance to the other end of the rod. Additionally, the same motion and steps could be performed on a vertical rod with the robot moving either up or down.

If the motion before or after the translational motion is a disassembly motion, the initial or final steps vary slightly. In the event that a disassembly just occurred, only one translational mechanism is engaged. If the robot moves to a rod with the intention to remove it, the robot will not engage the second translational mechanism but instead will start with the sequence to disassemble the rod.

2) Disassembly Motion: The disassembly motion starts with one-half of the robot connected to a horizontal rod via its

translational mechanism. The second half is at the rod to be disconnected. The robot then performs the steps as shown and described in Table 4.

When the rotational mechanism is engaged, feedback from the servos is used to gauge torque and rotation amount applied by the cam actuator. These readings are interpreted by the program to ensure proper rotational engagement has occurred. The robot will automatically realign, and re-engage itself until it has determined that the attachment will likely result in a successful disassembly.

3) Assembly Motion: For the assembly motion, the robot starts with one translational mechanism on one rod and the opposite rotational mechanism holding the rod to be connected to the node. The hinge is at an angle of 90°. The robot then performs the steps as shown and described in Table 4.

4) Disassembly of a Horizontal Rod, 90° Rotation, and Assembly at a Different Point of the Connector: This test was done to verify that a rod can be disconnected at one side of a connector and reconnected after a 90° rotation. This test was performed only once as a demonstration

7

86

5 49

31011

12 1

2

Figure 7. Robot motion in chairlike test structure.

(2) (3)(1)

(5) (6)(4)

(8)

(a)

(b)

(9)(7)

(2) (3)(1)

(5) (6)(4)

Figure 8. Disassembly and assembly motions: (a) vertical truss structure disassembly and (b) disassembly/assembly of vertical rod.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







that the system is robust enough to align itself correctly after the operation of the rotational mechanism. It is a starting point for allowing the robot to eventually reas-semble an entire structure independently. For this motion, the robot is connected at the vertical rod with the translational mechanism engaged. The hinge is set at 90°. The robot then performs the steps as shown and described in Table 4.

Disassembly Test Bed—Vertical SquareThe disassembly test bed is a vertical square plane in which the robot disassembles the top three rods of the structure. To disassemble the vertical structure, only a combination of the translational motions and the disas-sembly motions was necessary. The disassembly steps can be seen in Figure 8(a).

Assembly Test Bed—Single TrussThe assembly test bed consists of a single vertical rod. The rod was first unlocked and removed, then reattached and locked into place. The steps can be seen in Figure 8(b).

ResultsWe first examine the individual basic motions as described earlier and then display the overall results as obtained in the different test beds. The test beds are described in detail in the section “Test Bed.”

Robot R1 Traversal Test Results1) Basic Motions in Chairlike Test Structure: The results of

the chair traversal basic motion tests can be seen in Table 5. The overall success rates for the translational motion and the 90° vertical rotation were both above 90%. The success rate for the 180° horizontal rotation was only 42%. The average time for completion ranged from 11 to 14 s.

2) Overall Test Results in Chairlike Test Structure: Twelve tests were performed with two successful completions. This gave a success rate of 17% for the traversal of the complete structure. As can be surmised from the results of the basic movement tests, the majority of these failures on the chair structure were attributed to the incomplete 180° horizontal rotation. The other

errors were due to a broken structural part and a servo malfunction due to a program or servo error.

Robot R2 Assembly and Disassembly Test Results1) Basic Motions for Assembly and Disassembly Test: The dis-

assembly motions performed with robot R2 showed high success rates of 97% and 100% for the translational and disassembly motions, respectively. The assembly motion had a success rate of 70%.

2) Overall Test Results for Assembly and Disassembly Tests:Two different test beds were used for the assembly and disassembly tests. The assembly tests were demonstrated on a single vertical rod, as the robot currently does not possess a carrying mechanism. A total of 11 tests were conducted with seven successful completions of disassem-bly and reassembly. The rotational mechanism erroneous-ly disengaged three times before the assembly was completed. One time the translational mechanism was disconnected from the horizontal rod during disassembly of the vertical rod.Fourteen vertical square disassembly tests were per-

formed, with ten of the 14 being completed successfully. Importantly, none of the failures were the result of design or programming flaws. The first failure was because of the bat-tery running out of power. Therefore, tests 3–14 were per-formed with the power cable attached to the charging side of the battery (this had no impact on power delivered to the motors, as the supply voltage was regulated by the control board). Despite the battery being charged while performing the tests, another power failure was observed during test 12, likely the result of a loose power cable. During one of the tests, a rubber band had broken that was used to pull the mechanisms to the disengaged position. As for another test, the robot encountered an exception that resulted from the experimenter’s failure to remove a detached structural rod from the test structure.

DiscussionThe tests showed that only three basic motions are needed for the robot to reach any point in the chairlike structure. Two out of the three basic motions needed to traverse in any structure were performed at high success rates of over 90%. The errors for the 90° vertical rotation were due to a broken

Table 5. Basic motions results.

Movement in Chairlike Structure Disassembly TestsAssembly

Tests

Translational Motion

90° Vertical Rotation

180° Horizontal Translational Motion

Disassembly Motion

Assembly Motion

Attempts 56 26 12 129 46 10

Successes 55 24 5 125 46 7

Success rate (%) 98 92 42 97 100 70

Time average (s) 12 14 11 18 11 6

Time standard deviation 0.9 0.5 1.2 3.2 0.9 0.4


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







structural part. This part is identical on robots R1 and R2, and during all the tests, this was the only time it broke; it could easily be designed to be stronger with no impact on the function of the robot and very little increase in weight.

The 180° horizontal rotation is the weakest of the basic motions required for movement in the structure. The errors were due to incomplete attachment of both rotational mech-anisms. Fortunately, this is the least used motion, and both rotational motors were upgraded in R2 to alleviate this prob-lem. Automatic engagement checking was also implemented to reduce these errors in R2. To complete a path as shown in Table 4, a combination of 16 translational motions, four 90° vertical rotation motions, and two 180° horizontal rotations are needed. The 180° horizontal rotation motion is not need-ed for disassembling a vertical plane. In R2, the reflectivity sensors could theoretically be used to assist in the verifica-tion of proper structural engagement, though this has not yet been implemented.

For structural disassembly and assembly, the robot had to be upgraded from R1 to R2, which included the addition of sensors for determining translational stopping points. The disassembly of a vertical structure was well performed by this robot because of the added accuracy lent by feedback from these sensors. The disassembly motion was performed with a 100% success rate, because the sensors could be used to reli-ably detect the ideal stopping point which ensured proper truss engagement. The translational motion success rate of R2 was comparable with the one obtained using robot R1. This shows that the use of stronger servos and line detection sen-sors in robot R2 compensated for the flexibility introduced by the lockable connectors.

The time to finish the translational motion was higher in R2, but this was only because of a decreased speed assigned in the programming.

The lockable connectors added a large amount of flexibili-ty that resulted in a significant deflection from the vertical plane when the robot traversed up or down a single vertical rod. The 3-D printer cannot print the circular locking ele-ment accurately enough to ensure that tolerances are such that deflections are minimized. The deflection appears large but the connectors are strong enough to handle repeated abuse, as was demonstrated during the disassembly tests. The deflection angle can be seen in the demonstration video that accompanies this article.

R2 performed the assembly motion at a success rate of 70%. The errors observed are suspected to be the result of a communication error in the program. When the error is cor-rected, the success rate is expected to be above 90%, as was the case with the disassembly tests. Currently a design limita-tion is that the rods have to be assembled to the node from a specific direction.

The basic disassembly, 90° rotation, and reassembly of a horizontal rod were performed once for demonstration pur-poses. Its success confirms that the bidirectional gears and their self-centering ability are strong enough to reconnect a truss at new vertices of a connector element. This confirma-

tion opens up the possibility of creating a robot that can com-pletely rebuild a structure.

The array of tests performed validate that the robot has the unique capability to disassemble any structure and provide lim-ited reassembly options. As the current iteration of the robot is capable of reaching any point within the structure, complete disassembly is possible. Moreover, the rods that are needed for the construction of a structure, but not for the final design could be disassembled by the robot and reused. With the ability to traverse the structure in all directions and orientations, the robot could also be used to inspect a finished truss structure.

The biggest current hurdle for the robot is the absence of means for transporting rods across joints in the structure. This will be rectified in a future revision through the addi-tion of a carrying pod. Difficulties will include maintaining a reasonable mass to power ratio and ensuring that the robot still has sufficient range of motion. Should the robot be implemented in a zero-gravity environment such as the international space station, attention to mass constraints would not be as critical.

ConclusionIn this article, we showed the design of a robot that can move autonomously and untethered through a truss structure. The robot demonstrated continuous motion over a total of eight nodes in a chairlike test structure and can successfully perform disassembly and assembly actions on a reconfigurable struc-ture. These tests confirmed that the robot is capable of navigat-ing through a vertical and a horizontal plane and transitioning between these planes because any cubic test structure consists of only vertical and horizontal planes and transitions between the planes. Only three basic motions were needed to move within the structure. Our novel adaptable connectors ensured that the robot could reliably disassemble and assemble hori-zontal and vertical trusses.

A combination of the motions demonstrated in this arti-cle theoretically allows the robot to take apart any size struc-ture. For very large structures, the reliability of individual motions will need to be further improved; this can be accomplished by using our existing feedback system to allow the robot to assess itself more rigorously. A video motion tracking system, observing the structure, could also be employed to determine whether rods were removed or connected correctly and where errors occurred most com-monly. This would be used only for gathering data; future versions of the robot will implement independent feedback, without the use of an external sensing system.

A carrying pod will eventually allow the robot to remove a rod from one location, carry it to another, and attach it there. Additionally, a mechanism for autonomously adding nodes will be implemented to facilitate an effective construction scheme. Disassembly and assembly with the ability to carry rods to any point in the structure and to add nodes open the door to reconfigurable and self-healing structures.

Future work will focus on improving reliability through increased sensor data, improved control schemes, and


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







superior error-correction actions. A carrying pod, a node-attachment mechanism, improved reliability, and motion error detection will allow us to create a robot and structure capable of nearly limitless configurations.

AcknowledgmentThis work was supported in part by NSF EFRI Grant 0735953. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the sponsoring organization.

References[1] S. Skaff, P. Stariz, and W. L. Whittaker, “Skyworker: Robotics for space assembly, inspection and maintenance,” in Proc. Space Studies Inst. Conf., 2001, pp. 1–5.[2] S. Yun and D. Rus, “Self assembly of modular manipulators with active and passive modules,” in Proc. 2008 IEEE Int. Conf. Robotics Automation, pp. 1477–1482.[3] W. Dogget, “Robotic assembly of truss structures for space systems and future research plans,” in Proc. 2002 IEEE Aerospace Conf., vol. 7, pp. 3589–3598.[4] K. C. Galloway, R. Jois, and M. Yim, “Factory floor: A robotically reconfig-urable construction platform,” in Proc. 2010 IEEE Int. Conf. Robotics Automa-tion, pp. 2467–2472.[5] R. Aracil, R. J. Saltarén, and O. Reinoso, “A climbing parallel robot,” IEEE Robot. Autom. Mag., vol. 13, no. 1, pp. 16–22, Mar. 2006.[6] C. Balaguer, A. Gimenez, and M. Abderrahim, “ROMA robots for inspec-tion of steel based infrastructures,” Ind. Robot: Int. J., vol. 29, no. 3, pp. 246–251, 2002.[7] P. Chatzakos, Y. P. Markopoulos, and K. Hrissagis, “On the development of a modular external-pipe crawling omni-directional mobile robot,” Ind. Robot: Int. J., vol. 33, no. 4, pp. 291–297, 2006.[8] I. M. Chen and S. H. Yeo, “Locomotion of a two-dimensional walking-climbing robot using a closed-loop mechanism: From gait generation to navi-gations,” Int. J. Robot. Res., vol. 22, no. 1, pp. 21–40, Jan. 2003.[9] K. A. Daltorio, T. E. Wei, A. D. Horchler, L. Southard, G. D. Wile, R. D.Quinn, S. N. Gorb, and R. E. Ritzmann, “Mini-Whegs climbs steep surfaces using insect-inspired attachment mechanisms,” Int. J. Robot. Res., vol. 28, no. 2, pp. 285–302, Feb. 2009.[10] M. Elliott, W. Morris, A. Calle, and J. Xiao, “City-climbers at work,” in Proc. 2007 IEEE Int. Conf. Robotics Automation, pp. 2764–2765.[11] Y. Fu, Z. Li, and S. Wang, “A wheel-leg hybrid wall climbing robot with multi-surface locomotion ability,” in Proc. 2008 IEEE Int. Conf. Mechatronics Automation, pp. 627–632.[12] G. J. Goldman, “Design space and motion development for a pole climb-ing serpentine robot featuring actuated universal joints,” M.S. thesis, Dept. Mech. Eng., Virginia Polytech. Inst., Blacksburg, VA, 2009.[13] C. Hillenbrand, D. Schmidt, and K. Berns, “CROMSCI: Development of a climbing robot with negative pressure adhesion for inspections,” Ind. Robot: Int. J., vol. 35, no. 3, pp. 228–237, 2008.[14] L. P. Kalra, J. Gu, and M. Meng, “A wall climbing robot for oil tank inspec-tion,” in Proc. IEEE Int. Conf. Robotics Biomimetics, 2006, pp. 1523–1528.[15] B. Kennedy, A. Okon, H. Aghazarian, M. Badescu, X. Bao, Y. Bar-Cohen, Z. Chang, B. E. Dabiri, M. Garrett, L. Magnone, and S. Sherrit, “Lemur IIb: A robotic system for steep terrain access,” Ind. Robot: Int. J., vol. 33, no. 4,pp. 265–269, 2006.

[16] K. D. Kotay and D. Rus, “Navigating 3-D steel web structures with an inchworm robot,” in Proc. 1996 IEEE/RSJ Int. Conf. Intelligent Robots Systems,pp. 368–375.[17] S. P. Krosuri and M. A. Minor, “Design, modeling, control, and evaluation of a hybrid hip joint miniature climbing robot,” Int. J. Robots. Res., vol. 24, no. 12, pp. 1033–1053, Dec. 2005.[18] M. A. Minor and R. Mukherjee, “Under-actuated kinematic structures for miniature climbing robots,” J. Mech. Design, vol. 125, no. 2, pp. 281–291,Jun. 2003.[19] T. P. Sattar, H. L. Rodriguez, and B. Bridge, “Climbing ring robot for inspection of offshore wind turbines,” Ind. Robot: Int. J., vol. 36, no. 4,pp. 326–330, 2009.[20] M. J. Spenko, G. C. Haynes, J. A. Saunders, M. R. Cutosky, and A. A.Rizzi, “Biologically inspired climbing with a hexapedal robot,” J. Field Robot., vol. 25, nos. 4–5, pp. 223–242, 2008.[21] D. Sun, J. Zhu, C. Lai, and S. K. Tso, “A visual sensing application to a climbing cleaning robot on the glass surface,” Mechatronics, vol. 14, no. 10,pp. 1089–1104, 2004.[22] F. Tâche, W. Fischer, G. Caprari, R. Siegwart, R. Moser, and F. Mondada, “Magnebike: A magnetic wheeled robot with high mobility for inspecting complex-shaped structures,” J. Field Robot., vol. 26, no. 6, pp. 453–476, 2009.[23] M. Tavakoli, M. R. Zakerzadeh, G. R. Vossoughi, and S. Bagheri, “Ahybrid pole climbing and manipulating robot with minimum DOFs for construction and service applications,” Ind. Robot: Int. J., vol. 32, no. 2,pp. 171–178, 2005.[24] M. Tavakoli, A. Marjovi, L. Marques, and A. T. de Almeida, “3DCLIMB-ER: A climbing robot for inspection of 3-D human made structures,” in Proc. 2008 IEEE/RSJ Int. Conf. Intelligent Robots Systems, pp. 4130–4135.[25] T. S. White, R. Alexander, G. Callow, A. Cooke, S. Harris, and J. Sargent, “A mobile climbing robot for high precision manufacture and inspection of aerostructures,” Int. J. Robot. Res., vol. 24, no. 7, pp. 589–598, Jul. 2005.[26] Y. Yoon and D. Rus, “Shady3-D: A robot that climbs 3-D trusses,” in Proc. 2007 IEEE Int. Conf. Robotics Automation, pp. 4071–4076.[27] D. A. Hjelle, “Toward machine metabolism: design of a truss reconfigur-ing robot,” M.S. thesis, Sibley School Mech. Aerosp. Eng., Cornell Univ., Ithaca, NY, 2009.[28] C. Detweiler, M. Vona, Y. Yoon, S. Yun, and D. Rus, “Self-assembling mobile linkages,” IEEE Robot. Autom. Mag., vol. 14, no. 4, pp. 45–55, Dec. 2007.[29] Y. Terrada and S. Murata, “Automatic modular assembly system and its dis-tributed control,” Int. J. Robot. Res., vol. 27, nos. 3–4, pp. 445–462, Mar./Apr. 2008.

Franz Nigl, Cornell Creative Machines Lab, Cornell University, Ithaca, NY 14853, USA. E-mail: [email protected].

Shuguang Li, Northwestern Polytechnical University, Xi’an, Shaanxi 710072 and Cornell University, Ithaca, NY 14853, USA. E-mail: [email protected].

Jeremy Evan Blum, Cornell Creative Machines Lab, Cornell Uni-versity, Ithaca, NY 14853, USA. E-mail: [email protected].

Hod Lipson, Sibley School of Mechanical and Aerospace Engi-neering, Cornell University, Ithaca, NY 14853, USA. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_______________

___________

______________

____________











By Claudio Melchiorri, Gianluca Palli, Giovanni Berselli, and Gabriele Vassura

Overview of Design Solutions and Enabling Technologies

The replication of the human hand’s functionality and appearance is one of the main reasons for the development of robot hands. Despite 40 years of research in the field [1], the reproduction of human capabilities, in terms of dexterous

manipulation, still seems unachievable by the state-of-the-art technologies. From a design perspective, even defining the optimal functionalities of a robotic end-effector is quite a challenging task since possible applications of these devices span industrial robotics, humanoid robotics, rehabilitation medicines, and prosthetics, to name a few. Therefore, it is reasonable to think that the design solutions, which are well suited to a single domain, might not be readily taken as general guidelines. For example, industrial manipulators are often equipped with basic grippers, which are conceived so as

to increase the throughput and the reliability, and are assumed to operate in structured environments. In this case, the enhanced manipulation skills and the subsequent cost increases must be carefully motivated by the application requirements.

Robotic HandA different scenario arises when dealing with the fields of humanoid robotics and prosthetics (see, e.g., [2] and [3]), where the robotic end-effector is expected to provide high flex-ibility and adaptability, ideally replicating the overall function-ality of the human hand. The general perception, however, is that the current hands, especially those aiming at an anthropo-morphic appearance, are either too complex and expensive or too bulky and unreliable to truly represent effective solutions.

Within this scenario, this article discusses the main design issues faced by the authors while developing four generations of anthropomorphic robot hands. The recent activities for the


Date of publication: 31 May 2013

PH

OTO

CO

UR

TE

SY

OF

GIA

NLU

CA

PA

LLI A

T T

HE

LA

BO

RAT

OR

Y O

F A

UTO

MAT

ION

AN

D R

OB

OT

ICS

OF

TH

E U

NIV

ER

SIT

Y O

F B

OLO

GN

A


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







realization of the latest artificial hand, called the University of Bologna Hand, version IV (UB Hand IV, see Figure 1), are illustrated through a general overview of the main design aspects and related technologies. This article aims at suggesting some concrete solutions to the difficulties encountered so far.

Related Work and Possible Directions for ImprovementMany anthropomorphic robot hands have already been designed, often trying to replicate or enhance the specific features of the human hand. With the exception of a few direct-drive solutions, tendon transmissions are usually preferred because of the technological problems in the inte-gration of the driving system within the hand. For instance, in the Utah/Massachusetts Institute of Technology robot hand, two antagonistic tendons for each joint are used, whereas in the Jet Propulsion Laboratories/Stanford hand, an N 1+ tendon network was adopted to reduce the amount of actuators. Later, the sheath-guided tendons were adopted in the UB Hand III [4] and, recently, the Deutschen Zentrums für Luft- und Raumfahrt (DLR) developed a new tendon-driven robot hand with compliant actuators [5]. It is also worth mentioning the Shadow Hand as the only commercially available tendon-driven device. In addition, during the last ten years, technological advance-ments have facilitated the development of modular hands with integrated drive systems, such as the DLR Hand II [6]or the Twendy-One Hand. This solution allows the mini-mization of space requirements and the potential installa-tion of these hands as end-effectors in common industrial arms. The great interest in the development of anthropo-morphic robot hands is also due to their use as prostheses, and several devices for this application have been devel-oped, e.g., the Cyberhand [2] and the DLR/Harbin Institute of Technology prosthetic hand [3].

There have been dozens of different design proposals, which makes it difficult to compare and assess within this wide field. Nonetheless, in the authors’ opinions, the cur-rent design of robot hands is mainly based on methodolo-gies inherited from conventional mechanics and robotics. The current technology, in terms of materials, sensors, and motors’ dimensions and power, has somehow reached the limits, as far as possible solutions that can be proposed fol-lowing the conventional design approaches. A potential direction of improvement is to seek alternative solutions for the realization of the hand structure and the sensorim-otor system that are simple and, possibly, reliable. In par-ticular, the general goal of the design simplification and optimization might be achieved through the following driving issues:● trying to reduce the assembly complexity by adopting

endoskeletal structures articulated by means of noncon-ventional joints, either sliding or compliant (see the section “Design Solutions for the Finger Structure”)

● actuating the joints by means of remotely located actuators, with the tendon-based transmissions routed by sliding

paths (sliding tendons) and integrated within the finger structure (see the section “Tendons and Tendon Net”)

Figure 1. The UB Hand IV prototype. (Photo courtesy of Gianluca Palli at the Laboratory of Automation and Robotics of the University of Bologna.)

Figure 2. The UB Hand study prototypes: (a) endoskeleton principle (UB Hand III [4])—the outer layer hosts sensors and interacts physically with the environment, (b) exoskeleton principle (UB Hand II [7]), (c) finger with close-wound springs CJs, and (d) fully integral finger. (Photo courtesy of Gianluca Palli at the Laboratory of Automation and Robotics of the University of Bologna.)

(a) (b)

(c) (d)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







● employing easy-to-control compliant actuators that are suf-ficiently compact to be hosted in a human-sized forearm (see the section “Actuation”)

● employing an appropriate but simple sensory apparatus to enhance the hand functionality and to compensate for the side effects raised by the mechanical design choices (see the section “Sensorial Apparatus”)

● exhibiting surface compliance through a purposely designed soft cover that closely mimics human skin (see the section “Design Solutions for the Hand Soft Cover”).These design approaches, described in the following text,

are adopted for implementation in the UB Hand III [see Fig-ure 2(a)] and are now exploited within the DEXMART proj-ect (http://www.dexmart.eu/). The last outcome of the project is a novel robotic hand, the UB Hand IV, whose prototype is shown in Figure 1.

Design Solutions for the Finger StructureMany solutions presented in the past [e.g., UB Hand II [7],Figure 2(b)] were inspired by exoskeletal models, the load-car-rying frame being a hollow structure characterized by closed cross sections with good stiffness/weight ratios. Nonetheless, this traditional design approach leads to a poor exploitation of the available space inside the finger, which is mainly used to host the articulations and the transmissions. A different, bio-inspired concept is based on the endoskeletal model and shows a functional distinction between an inner, stiff frame-work (the bones) and an outer, compliant layer (the flesh). This design solution allows space to be saved for hosting the sensors, the related electronics, and the pads, which are now placed around the articulated structure rather than inside it.

As for the finger joints, the main design goal is to search for the maximum achievable integration between the various components, in the perspective of structural simplification, potentially leading to one-step monolithic manufacturing and consequent reduction of the assembly complexity. In particu-lar, the use of compliant joints (CJs) or sliding kinematic pairs has been investigated by the authors in the last few years [4],[8] as possible alternatives to the classical rotational joints based on bearings and similar hardware. As examples of the above concepts, several solutions are explored along with the development of different UB hand prototypes:● modular fingers composed of plastic phalanges (obtained

by injection moulding) and CJs shaped as close-wound alloy springs [Figure 2(c)]

● monolithic fingers manufactured in Teflon by computer numerical control (CNC) machining, with CJs shaped as notch hinges [also, the tendons are integrated into the structure; see Figure 2(d)]

● monolithic fingers with integral CJs made of the same material (Fullcure 720) as the phalanx structure [Fig-ure 3(a)]

● fingers with pin joints integrated into the phalanx body simply consisting of a plastic shaft that slides on a cylindri-cal surface [Figure 3(b)].Concerning the integrated pin joints, despite the sliding

contacts, the joints can withstand an indefinitely large number of opening–closing cycles at a tendon tension of about 80 N. On the other hand, stiction and dynamic friction deteriorate the open-loop position control of the finger and can lead to the mechanism locking as the contact pressure between the shaft and the hub increases (due to increased tendon trac-tion). Nonetheless, the adoption of a sensory system and suit-able control strategies [9] can compensate for such side effects.

Concerning the CJs, their benefits when compared to tra-ditional kinematic pairs include the absence of wear, backlash, and friction, while still ensuring size and weight reduction. On the other hand, critical issues of the CJs are possible fatigue failure and undesired displacements. In fact, restricting the analysis to single degree-of-freedom (DoF) rotational CJs, these devices are conceived to allow a principal displacement along a desired reference direction (called compliant, see, e.g., [10]) when subjected to a principal load (torque or force) act-ing along the same direction. The ratio between the principal displacement and the principal load is called the principal compliance. The secondary or parasitic displacements along the other reference directions may occur in real applications both for the presence of the secondary or parasitic loads acting along those directions and for the presence of a compliant axis drift. As for the axis-drift, even in the absence of secondary loads, the point that models the center of rotation of the CJs can be subjected to a spatial motion during joint deformation. In any case, both fatigue failures and secondary displacements are heavily dependent on the inseparable binomial material morphology, the achievable joint shapes being directly con-nected to the manufacturing technology. For instance, the CJs with relatively simple geometries, such as the beam-like

Figure 3. The UB Hand IV finger prototypes: (a) integrated CJs and (b) pin joints integrated into the phalanx. (Photo courtesy of Gianluca Palli at the Laboratory of Automation and Robotics of the University of Bologna.)

(a) (b)

Figure 4. The large-displacement rotational CJs: (a) SPIR CJ and (b) HEL CJ.

z

y

xl

O

z

y

xl

O

(a) (b)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=http://www.dexmart.eu/&id=18410&adid=P74E1






flexures depicted in Figure 2(d), can withstand infinite load cycles and can be used to produce the monolithic fingers. In the same way, the close-wound spring CJs depicted in Figure 2(c), also withstanding infinite load cycles (at a tendon tension of about 80 N), are less sensitive to parasitic motions but unsuitable for the production of one-piece compliant structures. Recalling that the main design goal aims at reduc-ing the assembly complexity, at present, different technologies and a wide range of materials (including the lightweight metal alloys) can be used to produce the articulated finger structure in a single production step (fully integral finger). Such tech-nologies include the aforementioned CNC machining, plastic molding (such as shape deposition manufacturing [11]), selec-tive laser sintering, fused deposition modeling, stereolithogra-phy, and electron beam melting. Nevertheless, recent advances in plastic material technology suggest that the use of polymers might be well suited for the production of artificial hands when a lightweight, relatively economical solution is sought. In practice, a proper choice of materials and construction pro-cesses may lead to the production of CJs directly integrated into the rigid links. In this case, one can obtain articulated structures that are compact, light, safe, robust to impact, inex-pensive, and slender (such that the external sensors and the compliant covers are easily integrated).

Hence, the real design challenge is to determine the best CJ morphology that allows the desired principal displacement while minimizing parasitic effects. In particular, aiming at a compact CJ design and looking at the minimization of the parasitic effects, two large-displacement CJ morphologies are investigated by the authors [10]. Both these types of joints, as depicted in Figure 4, are characterized by the same specifica-tions in terms of range of motion, dimensions, principal com-pliance, material, and production technology. More details about comparison metrics among different CJ morphologies are reported in “CJs Comparison Criteria.” In particular, these joints are employed in the realization of the compliant finger shown in Figure 3(a), which is made of FullCure 430 Durus White, a photosensitive resin with mechanical properties sim-ilar to polypropylene. The reliability and fatigue life are still an issue. Nonetheless, further testing is in progress concerning fingers made with high-performance thermoplastic materials (such as Stratasys Ultem 9085).

Design Solutions for the Sensorimotor System

Tendons and Tendon NetThe current technology does not allow the arrangement of 20 or more actuators in a human-sized robot hand while meet-ing speed and force requirements. For this reason, it is often necessary to remotely place the actuators. In particular, as pre-viously discussed, the tendon-based transmission systems represent one promising solution toward the implementation of dexterous anthropomorphic robotic hands. Besides this consideration, it should be noted that the tendons are also present within the human hand, with the purpose of connect-ing bones and transmitting forces. Due to the relatively com-

plicated human tendon network, rather than directly imitating the biological model, many different simplified solu-tions are proposed in the literature. In fact, the optimization of the transmission system, in terms of friction reduction and decoupling among hand movements, requires the tendons to traverse the endoskeleton or to be routed close to the center of rotation of the joints by means of suitable canals (whereas in the biological models, the tendons slide around the bones). The well-known and effective configurations are [12]:● 2N configuration: Each

joint is actuated by two independent tendons. This configuration allows the independent control of the joint torques and the regulation of the internal forces but requires 2N actuators.

● N configuration: Each joint is actuated by a tendon con-nected in a loop to the actuator. This configuration allows the independent control of each joint, but it requires a mechanism for the tendon pretension.

● 1N+ configuration: The joints are actuated by a tendon net composed by a number of tendons equal to the num-ber of joints plus one. This configuration allows the use of the minimum number of actuators and, simultaneously, allows the avoidance of pretension mechanisms.

● Unilateral tendon actuation: This configuration exploits the energy stored in the CJs during the closing phase to per-form the opening phase without requiring further energy from the actuation [4], [11].

In tendon actuation systems, different movements can be eas-ily coupled by simply connecting two or more tendons to the same actuator, thus obtaining a defective actuation, avoiding additional mechanisms, and reducing both the costs and the complexity.

Another important problem in the tendon-based actua-tion is the routing of the tendons from the motors to the joints. Usually, the tendons are routed by means of pulleys, sheaths, or sliding surfaces, and pulleys reduce the friction forces at a minimum level by acting along the tendon. This approach implies a more complicated mechanical design, due to the presence of bearings and similar hardware par-tially reducing the advantages introduced by the use of ten-dons. The use of sheaths is a convenient solution due to its simplicity, but it introduces distributed friction along the ten-don, which means hysteresis and dead zones in the transmis-sion system characteristics [9]. The selection of the tendon material plays a crucial role. Usually, very thin steel ropes are used. This solution allows for the linear force-elongation behavior of the tendon, but it introduces some design and assembly constraints because of the limited curvature radius

Many anthropomorphic

robot hands have already

been designed, often trying

to replicate or enhance

the specific features of the

human hand.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







CJs Comparison CriteriaIn the present implementation, the helical (HEL) joint connects the proximal phalanx to the palm whereas the spiral (SPIR) joint connects the medial and distal phalanges. The proposed large-displacement CJs are compared by defining a set of compliance matrices, composed by entries of the nonuniform physical dimensions, which describe how the CJ reacts to external wrenches. Naturally, being a differential operator, each matrix measures only local configuration-dependent CJ properties. Hence, the comparison is based on a set of normalized, dimensionless local matrices [C]k, evaluated by means of FEM within the CJ workspace. (Having defined I l/EC C* ,M Myy y y= i i being the CJ principal compliance, displacements are normalized by the overall joint length L[Figure 4(a)], forces by */EI L2 , and moments by */EI L .) Given the normalized dimensionless loads vector f3 acting on a reference point, the normalized dimensionless joint displacement u3 is expressed by

u C f3 $3=

u u u u[ ]x y z x y zT3 3 3 3 3 3 3i i i=

f f f f m m m ,[ ]x y z x y zT3 3 3 3 3 3 3= (S1)

where the translational and rotational components along the reference axis of both the load and the displacement vector are specified. The general structure of the compliance matrix is

C

CCCCCC

CCCCCC

CCCCCC

CCCCCC

CCCCCC

CCCCCC

26

11

21

31

41

51

61

12

22

32

42

52

62

13

23

33

43

53

63

14

24

34

44

54

64

15

25

35

45

55

65

16

36

46

56

66

=

R

T

SSSSSSSS

V

X

WWWWWWWW

. (S2)

In ideal revolute CJs, pure rotation along the principal direction (the y axis) is expected. Hence, its normalized compliance matrix is characterized by a finite unit entry in C55,all the other entries being null. The term C55 is reported in (S2) and arises from a rotation around the principal axis under the action of the load my . On the contrary, a real CJ will present finite values of normalized compliance coefficients along the other directions.

To evaluate the compliance matrices in the different joint configurations and assuming that SPIR and HEL joints are capable of 90° rotation 45! °, the joint workspace is divided into N 5= equally spaced intervals, as shown in Figure S1, where 0° represents the joint undeformed configuration. Then, a fraction of the maximum principal load my is applied to reach the kth (where , ...,k N1= ) configuration, and a small

variation of the load vector f3 is applied while maintaining the previous principal load. As a result, the related joint displacement u3 and the related compliance matrix can be evaluated. Each matrix is then split into two submatrices CR and CT containing, respectively, the coefficients relative to angular and linear displacements along the reference directions:

, ,C C C C C C C, ,k RT

TT

kT

Rk i j Tk l j= = =6 @, ..., , , ..., , , ..., .i l j1 3 4 6 1 6= = = (S3)

The two local performance indexes (LPIs) that characterize the joint performance in each configuration, specifically a rotational LPI, IR, and a translation LPI, IT, are evaluated by using the weighted Frobenius norm [10] (see Figure S2):

,I C I CRk Rk F Tk Rk F= = . (S4)

A smaller LPI indicates a better local CJ behavior. Starting from this local evaluation of the joint behavior, global performance indexes (GPIs), which summarize the overall joint performance over the whole workspace are defined and evaluated,

,I N

II N

Ig

R

Rkk

N

gT

Tkk

N

1 1= == =

/ /. (S5)

A smaller GPI indicates a better global CJ behavior. Note that small secondary rotations at joint level can be dramatically amplified at the end of serial articulated chains, hence the evaluation of Ig

R is usually more significant. The GPIs obtained from the analysis are .I 9 88 10g

T3$= - and

.I 1 00 10gR

2$= - concerning the SPIR CJ, and .I 1 59 10gT

2$= -

and .I 1 60 10gR

2$= - concerning the HEL CJ.

Figure S2. LPI trend for joints SPIR and HEL [Figure 4(a) and (b)] (a) Trend of LPI IR for SPIR and HEL. (b)]. Trend of LPI ITfor SPIR and HEL.

500-500.06

0.08

0.1

0.12

0.14

0.16

Principal Angular Task

(a)

500-50Principal Angular Task

(b)

Rot

atio

nal L

PI

0.350.4

0.450.5

0.550.6

0.650.7

Tra

nsla

tiona

l LP

I

SPIRHEL

SPIRHEL

Figure S1. The FEM models and workspace discretization: (a) SPIR joint and (b) HEL joint.

(a) (b)

-45°-22.5°

+45°+22.5°0°

-45°-22.5°

+45°+22.5°0°


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







of steel cables. In the last few years, polymeric fibers are largely used to improve the design flexibility of the tendon transmissions. Despite the comparable elastic module with respect to the alloy cables, the polymeric tendons present hysteresis in the force-elongation characteristic that intro-duces stability problem in the transmission system control [9]. Different studies confirm that the total amount of fric-tion acting along the tendon depends only on the friction coefficient and on the total curvature of the tendon path from the motor to the joint [9]. While the friction coefficient can be reduced by a suitable selection of the path coating and tendon materials (other than introducing lubricants), the path curvature minimization is a nontrivial design problem. As a case study, a detailed view of the 1N+ type tendon network of the UB Hand IV is presented in Figure 5. In par-ticular, Figure 5(a) shows the straight path of the tendons from the finger base to the wrist center, to avoid tendon cur-vature inside the palm (no friction is introduced) and mini-mize the coupling between the wrist and the finger movements. The optimization of the tendon path inside the finger is shown in Figure 5(b). In this figure, the tendons that actuate the base joint (T1 and T2) are connected directly to the proximal phalanx, whereas the antagonistic tendon (T4) slides over the joints on the back finger side. The tendon that actuates the medial joint (T3) is routed very close to the cen-ter of the proximal joint to limit, as much as possible, the coupling between the movements of these two joints. The path of the tendon (T5) that connects the proximal to the medial joint is straight to avoid contact between the tendon and the endoskeleton and to limit friction. Finally, Figure 5(c) shows a 3-D view of the finger and a detail of the joints’ locations. Note that, to reduce the path curvature, part of the tendon path is formed directly within the endoskeletal struc-ture of the finger. The chosen tendon material is Dynema

Fast-Flight (friction coefficient .0 12n = ), whose capability to accomplish a minimum of 100,000 working cycles under different load conditions is tested.

ActuationSignificant effort is currently being devoted to the develop-ment of innovative actuation strategies, e.g., distributed macro-mini [13], series-elastic [14], and variable-stiffness [15] actuation, as well as innovative actuators based on smart materials such as shape-memory alloy [16], electroactive

Figure 5. The UB Hand IV: details of the tendon network: (a) Frontal view of the tendon path inside the hand, (b) lateral view of the tendon path inside the finger, and (c) 3-D view of the finger design. (Photo courtesy of Gianluca Palli by means of CAD.)

(b)

T3T4

T5

Straight Path

TendonSlidingPoints

(a)

T1 T2

Straight Path

WristCenter

To theActuation

(c)

DistalJoint

MedialJoint

ProximalJoint

(2DoF)

Figure 6. (a) The basic concept and (b) a schematic representa-tion of the twisted-string actuation system.

0

4r

8r

12r

50.00

48.40

43.37

33.10

TwistAngle(rad)

ActuationLength (mm)

Encoder

dc Motor(Rotative)

OutputShaft

AxialBearing

(a)

(b)

ActuationFrame

Twisted String

Load Cell

i


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







polymers [17], and piezoelectric actuators [18], but these technologies are still far from implementation in anthropo-morphic robot hands.

The actuation of the commercial robotic hands developed so far is based essentially on rotative electric motors or linear pneumatic actuators, usually McKibben motors. The use of pneumatic actuators (besides the higher power density with respect to electric motors) presents some difficulties from the control point of view and also conflicts with the integration requirements. In fact, the valves and the compressor cannot be included in the hand structure because of their dimensions and weight. Concerning robotic hands with electric actuators, different types of transmission systems are adopted: the motors (usually dc or brushless motors) can be directly placed within the fingers and the palm and connected to the joint by means of a harmonic drive [6] or spur and worm gears [19],[20]; the motors can be placed in the forearm, transmitting the mechanical power by means of flexible shafts [21] or leverages or, as previously stated, the tendons routed by means of pulleys [7] or sheaths [4], [22].

Considering the problems related to integration and size, rotative electric motors are now the best technological solu-tion for actuation. Hence, conversion from rotative to linear motion is needed in the case of tendon-based transmission. Usually, this problem is solved by means of pulleys or ball screws connected to the motor gearbox. These solutions, even if reliable and effective, can cause some problems regarding the dimensions and the costs. Moreover, commercial minia-turized electric motors present the best efficiency at very high speed and very limited torque. Hence, the adoption of gear-boxes with high reduction ratio and low friction is needed, thus increasing the complexity, dimensions, and costs of the actuation unit. To solve these problems, the so-called twisted-string actuation system [23] is developed. With respect to conventional solutions, the main advantages of this actuation system are: 1) the direct connection between the motor and the tendon without intermediate mechanisms (such as gear-boxes, pulleys, or ball screws), 2) the direct transformation from rotative to linear motion, 3) the extremely reduced fric-tion (only an axial bearing is needed), 4) the very high reduc-tion ratio, 5) its intrinsic compliance, and 6) the possibility of using very small high-speed motors. The basic concept of this actuation system is depicted in Figure 6: the overall length of the transmission is reduced by twisting the tendons at one end by means of the motor, resulting in a linear motion of the other tendon end. Figure 6 also depicts the scheme of a twisted-string transmission system actuating a single finger, whereas Figure 7 shows a prototype of the proposed device. In particular, the twisted-string transmission is tested over more than 10,000 working cycles at a tendon tension of 40 N, showing no significant variation on the behavior of the sys-tem [23]. This actuation modality allows us to obtain a very compact and lightweight actuation module as can be seen in the computer-aided design (CAD) of the hand prototype shown in Figure 8 and composed of 24 actuators easily located in the forearm.

Figure 7. The finger prototype actuated by a twisted-string actuator: (a) detailed view of the robotic finger, (b) detailed view of the motors of the twisted string actuators, and (c) the robotic finger and its actuation module. (Photo courtesy of Gianluca Palli at the Laboratory of Automation and Robotics of the University of Bologna.)

(a) (b)

(c)

Figure 8. The UB Hand IV virtual prototype including 24 twisted-string actuators. (Photo courtesy of Gianluca Palli by means of CAD.)

24 Twisted-StringActuators

Twisted String (Red)+

Tendon Network (Green)

Figure 9. (a)–(d) The preliminary prototypes of the UB Hand IV sensors based on optoelectronic components. (Photo courtesy of Gianluca Palli at the Laboratory of Automation and Robotics of the University of Bologna.)

(a) (b)

(c) (d)

LED

Photodiode

8,969 mm

11,308 mm


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Sensorial ApparatusThe introduction of the compliant structures and the trans-mission systems based on the tendons, together with the use of innovative actuation systems like the twisted-strings actua-tion, requires the use of appropriate sensory equipment and the adoption of suitable control strategies. These strategies become necessary for dealing with the intrinsic system com-pliance and for compensating for the side effects, such as non-linearities and friction, which result as a consequence of the aforementioned design choices. The underlying philosophy is basically to attempt shifting the system complexity from the time-consuming domain of mechanical design to the easily reprogrammable domain of control. Generally speaking, a robotic hand can be equipped with position/velocity sensors for the measurement of the joints’ and/or actuators’ configura-tion, with force sensors for measuring the force applied on the wrist, the palm, the phalanges, and/or the joints and tactile sensors for reconstructing the pressure map during the con-tact (usually restricted to the fingertip) by means of an array of sensitive elements. In some limited cases, proximity and single pressure sensors are also part of the robot hand sensory equipment. For many reasons, to improve the reliability and to simplify the sensory subsystem, it is preferable to use sensi-tive elements with intrinsic high immunity to electromagnetic disturbances and with limited requirements in terms of both conditioning electronics and amplification. This is the main reason why the use of optoelectronic components is studied for all the sensors needed in the robot hand. In particular, an LED–photodetector couple with a wide angle-of-view is adopted for the implementation of the joint position sensors [24] depicted in Figure 9(a) [resolution of about (1/100)°], whereas components with a narrow angle-of-view are used for the force sensors [25] shown in Figure 9(b) and (c) (reso-lution of 0.05 N). Many different principles can be used for the implementation of the tactile sensors, but the problem is still the integration of all the electronics and acquisition sys-tem within the limited space of the fingertip. To solve this issue, a tactile sensor based on discrete surface-mount device optoelectronic components is developed [26], and in Figure 9(d), the grid of 4 4# taxels of the sensors prototype is shown. This sensor allows direct data acquisition (without any amplification circuit) about the deformation of the soft pads mounted above the sensor grid. For example, if a plane is

pushed against the soft cover, the tactile sensor can detect the overall reaction force with a resolution of 0.2 N and the plane position and orientation (with respect to the sensor center) with a resolution of 100 μm and 0.2°, respectively [27].

Design Solutions for the Hand Soft CoverThe soft covers are introduced as they improve functionality, safety, and acceptance by the users. The majority of soft pads studied so far were made by viscoelastic polymers homoge-neously shaped over an internal rigid core mimicking the bone or the robotic finger inner structure. In such a case, for a given external geometry, the parameters that contribute mainly to the pad compliance are the material hardness and the layer thickness [28]. A higher thickness implies higher compliance, which is beneficial in terms of safety and grasp stability/sustainability. On the other hand, high pad thickness signifies an undesired increase of limb dimension. In parallel, a higher material hardness, which is beneficial in terms of the surface reliability, signifies an undesired lower compliance. In practice, thickness reduction for a given compliance becomes a significant design goal as long as, usually, the adopted solu-tions trade off between the need of slender robotic limbs and good material properties.

Looking for alternative solutions to homogeneous soft covers, the concept of differentiated layer design (DLD) (which allows increasing the pad’s compliance while mini-mizing its thickness) is proposed by the authors [28]. This concept consists of the adoption of a single material dividing the overall thickness of the pad into layers with different structural designs (i.e., an external continuous skin layer cou-pled with an internal layer with voids). Figure 10 shows a DLD soft pad, whereas Figures 11 and 12 show the pad’s internal and external layers.

In particular, the pad quasistatic relation between the applied normal load, F, and the contact deformation, d[load-deformation (LD) curve] can be described as a nonlin-ear function: ( )F f d= . Given the pad thickness and material, the overall contact area can then be split into finite elemen-tary triangular sub-regions or triangular elements (TE) defined by tiling the plane regularly with equilateral triangles as in Figure 11. Once the element LD curve is designed to obtain the desired nonlinear characteristic, the number of elements, N, can be chosen such that

( )F N ft$ d= , (1)

where ( )ft d is the element LD curve. Naturally, the pro-posed procedure concerns the definition of an overall pad

Figure 10. The DLD concept.

Rigid Core

Indenter (External Surface)

Soft Pad

The main design goal

is to search for the

maximum achievable

integration between the

various components.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







contact force (i.e., overall pad compliance), which is an inte-gral (rather than a local) property of the pad. In addition, we suppose that N elements are involved in the contact simulta-neously and the contact area is displacement independent. To replicate the compression law of the human pulps (Fig-ure 13) while reducing pad thickness, the TE internal layer might be designed using a series of microbeams inclined with respect to the orthogonal to the external surface (nor-mal axis, Figure 12), thus transforming normal loads acting on the contact into bending actions applied on each beam. The microbeams are placed on the edge of the TE, as shown in Figure 12. The TE dimensions t, h, k, and j in Figure 12are found by trial and error with the aid of the finite element

method (FEM) and aim at replicating the human finger experimental LD curves shown in Figure 13. Note that the FEM simulations are carried out by imposing a gradual displacement to the indenter along its normal axis and by measuring the related reaction force’s

variation. Similarly to the human pulp, this peculiar TE geometry presents a quasilinear LD curve characterized by a very low stiffness for small displacements, the load rapidly increasing once the microbeams collapse (Figure 12) on the outer skin. In such a situation, the TE behaves similarly to a pad made of a uniform soft material. In the actual imple-mentation, the pad thickness is set to 3 mm (i.e., half of the

thickness of previously published solutions [4]), whereas the TE final design (Figure 11) is characterized by mm.t 0 5= ,

mmh 2= , mmk 1= , and 45ci = . The triangle facet, l, is chosen on the basis of (1), and the TE material is Tango Plus Fullcure 930 (hardness 27, Shore A), a polymeric resin used for stereolithographic prototyping processes. For instance, considering the distal phalanx, the contact area is a

mm20 15 2# rectangle meshed by means of 36 TEs. The resulting surface area of a single TE is . mm8 3 2 . Figure 13shows the numerical (FEM) or experimental (exp.) relation-ship between the normal load (N) and the resulting dis-placement (mm) for 1) the structured pad prototype depicted in Figure 11, 2) a uniform PAD of the same thick-ness (3 mm) made of a softer material, and 3) the human finger. It can be seen that a 3-mm-thick structured pad rep-resents a substantial step forward in human finger mimicry in terms of stiffness, when compared to previously pub-lished solutions where different materials and higher pad thicknesses are used. Similar procedures but different con-tact areas and numbers of TEs are adopted for the design of the medial and proximal pads, whose characteristics are also reported in Figure 13. The potentialities of the DLD concept are experimentally evaluated on hemispherical soft pads

shaped over a rigid core [28]. The pads to be mounted on the hand prototype are shaped as in Figure 10,and their physical implementation is shown in Figure 1.

Conclusions and PerspectivesIn this article, an overview of the design solutions and enabling tech-nologies along with some possible directions for improving the design of the robot hands are discussed. In particular, considering the numerous projects developed in the past and

Figure 13. The displacement (mm) versus normal load (N) for a DLD pad and a human fingertip.

Distal (exp.)Medial (exp.) DLD Pad (FEM)

Unform Pad (exp.)Proximal (exp.)5

4

3

2

1

0

Displacement (mm)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Nor

mal

Loa

d (N

)

Figure 11. A triangular grid.

Contact Area Internal Layer

Figure 12. The model mesh and deformed TE (collapsed beam): (a) detailed view of the TE, (b) TE model mesh, and (c) deformed TE.

Normal Axis Indenter ImposedDisplacement

1

h

t i

k

InternalLayer

ExternalLayer

(a) (b) (c)

Even defining the optimal

functionalities of a robotic

end-effector is quite a

challenging task.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







comparing their features with those of the reference model, the human hand, it is evident that evolution is still necessary.

The impression is that the main problems are not due to technological aspects but come from the design methodolo-gies inherited from industrial robotics or conventional mechanics. Innovative approaches based on nonconvention-al structures might be investigated to simplify the design, reduce the costs, and possibly enhance the performance of future robotic hands. In addition, off-the-shelf actuators and sensors that can be directly integrated in anthropomorphic hands do not exist, meaning that suitable solutions must be investigated.

In light of this consideration, the authors have reviewed, on the basis of their experience, some recent solutions con-cerning the finger structure, sensorimotor system, and soft cover of the hand. The underlying philosophy is to pursue a mechanical design that simplifies the hand manufacturing and assembly procedure. The side effects of these design choices are then compensated for through an appropriate sensory apparatus fully integrated within the hand. The pre-liminary prototype of the UB Hand IV, developed with avail-able technology, is also presented as an example of how the proposed solutions can be integrated in a general framework of reciprocal compatibility.

References[1] C. Melchiorri and M. Kaneko, “Robot hands,” in Handbook of Robotics, B.Siciliano and O. Khatib, Eds. New York: Springer-Verlag, 2008, pp. 345–360.[2] M. Carrozza, G. Cappiello, S. Micera, B. Edin, L. Beccai, and C. Cipriani, “Design of a cybernetic hand for perception and action,” Biol. Cybern., vol. 95, no. 6, pp. 629–644, 2006.[3] H. Huang, L. Jiang, Y. Liu, L. Hou, H. Cai, and H. Liu, “The mechanical design and experiments of HIT/DLR prosthetic hand,” in Proc. Int. Conf. Robotics Biomimetics, Dec. 2006, pp. 896–901.[4] L. Biagiotti, F. Lotti, C. Melchiorri, G. Palli, P. Tiezzi, and G. Vassura, “Development of UB hand 3: Early results,” in Proc. Int. Conf. Robotics Auto-mation, 2005, pp. 4488–4493.[5] M. Grebenstein, A. Albu-Schäffer, T. Bahls, M. Chalon, O. Eiberger, W. Friedl, R. Gruber, S. Haddadin, U. Hagn, R. Haslinger, H. Hoppner, S. Jorg,M. Nickl, A. Nothhelfer, F. Petit, J. Reill, N. Seitz, T. Wimbock, S. Wolf, T.Wusthoff, and G. Hirzinger, “The DLR hand arm system,” in Proc. Int. Conf. Robotics Automation, 2011, pp. 3175–3182.[6] J. Butterfass, M. Grebenstein, H. Liu, and G. Hirzinger, “DLR-Hand II: Next generation of a dextrous robot hand,” in Proc. Int. Conf. Robotics Auto-mation, 2001, vol. 1, pp. 109–114.[7] C. Melchiorri and G. Vassura, “Mechanical and control issues for integra-tion of an arm-hand robotic system,” in Experimental Robotics, vol. 190. Ber-lin, Germany: Springer-Verlag, 1993, pp. 136–152.[8] F. Lotti and G. Vassura, “A novel approach to mechanical design of articu-lated finger for robotic hands,” in Proc. Int. Conf. Intelligent Robot Systems,2002, vol. 2, pp. 1687–1692.[9] G. Palli, G. Borghesan, and C. Melchiorri, “Modeling, identification and control of tendon-based actuation systems,” IEEE Trans. Robot., vol. 28, no. 2,pp. 277–290, 2012.[10] G. Berselli, M. Piccinini, and G. Vassura, “Comparative evaluation of the selective compliance in elastic joints for robotic structures,” in Proc. Int. Conf. Robotics Automation, 2011, pp. 4626–4631.[11] A. Dollar and R. Howe, “The SDM hand: A highly adaptive compliant grasper for unstructured environments,” in Experimental Robotics (Springer Tracts in Advanced Robotics), vol. 54, O. Khatib, V. Kumar, and G. Pappas, Eds. Berlin, Germany: Springer-Verlag, 2009, pp. 3–11.

[12] L. Barbieri and M. Bergamasco, “Nets of tendons and actuators: An anthropomorphic model for the actuation system of dexterous robot hands,”in Proc. Int. Conf. Advanced Robotics, 1991, vol. 1, pp. 357–362.[13] M. Zinn, B. Roth, O. Khatib, and J. Salisbury, “New actuation approach for human- friendly robot design,” Int. J. Robot. Res., vol. 23, nos. 4–5, pp. 379–398, 2004.[14] D. Paluska and H. Herr, “The effect of series elasticity on actuator power and work output: Implications for robotic and prosthetic joint design,” Robot. Autonom. Syst., vol. 54, no. 8, pp. 667–673, 2006.[15] G. Palli, G. Berselli, C. Melchiorri, and G. Vassura, “Design of a variable stiff-ness actuator based on flexures,” J. Mech. Robot., vol. 3, no. 3, p. 034501, 2011.[16] S. Dutta and F. Ghorbel, “Differential hysteresis modeling of a shape memory alloy wire actuator,” IEEE/ASME Trans. Mechatronics, vol. 10, no. 2,pp. 189–197, Apr. 2005.[17] R. Vertechy, G. Berselli, V. P. Castelli, and G. Vassura, “Optimal design of lozenge-shaped dielectric elastomer linear actuators: Mathematical procedure and experimental validation,” SAGE, J. Intell. Material Syst. Struct., vol. 21, pp. 503–515, Mar. 2010.[18] S. Salisbury, D. Waechter, R. Mrad, S. Prasad, R. Blacow, and B. Yan, “Design considerations for complementary inchworm actuators,” IEEE/ASME Trans. Mechatronics, vol. 11, no. 3, pp. 265–272, June 2006.[19] W. T. Townsend, “The BarrettHand grasper programmably flexible part handling and assembly,” Ind. Robot, vol. 27, no. 3, pp. 181–188, 2000.[20] H. Kawasaki, T. Komatsu, and K. Uchiyama, “Dexterous anthropomor-phic robot hand with distributed tactile sensor: Gifu hand II,” IEEE/ASME Trans. Mechatronics, vol. 7, no. 3, pp. 296–303, 2002.[21] C. Lovchik and M. Diftler, “The robonaut hand: A dexterous robot hand for space,” in Proc. Int. Conf. Robotics Automation, 1999, pp. 907–912.[22] G. Berselli, G. Borghesan, M. Brandi, C. Melchiorri, C. Natale, G. Palli, S.Pirozzi, and G. Vassura, “Integrated mechatronic design for a new generation of robotic hands,” in Proc. IFAC Symp. Robot Control, 2009, vol. 9, pp. 105–110.[23] G. Palli, C. Natale, C. May, C. Melchiorri, and T. W¨urtz, “Modeling and control of the twisted string actuation system,” IEEE/ASME Trans. Mechatron-ics, vol. 18, no. 2, pp. 664–673, 2013.[24] A. Cavallo, G. De Maria, C. Natale, and S. Pirozzi, “Optoelectronic joint angular sensor for robotic finger,” Sens. Actuators A: Phys., vol. 152, no. 2, pp. 203–210, 2009.[25] G. Palli and S. Pirozzi, “Force sensor based on discrete optoelectronic components and compliant frames,” Sens. Actuators A, Phys., vol. 165, no. 2,pp. 239–249, 2011.[26] G. De Maria, C. Natale, and S. Pirozzi, “Force/tactile sensor for robotic applications,” Sens. Actuators A, Phys., vol. 175, pp. 60–72, Mar. 2012.[27] G. Palli, C. Melchiorri, G. Vassura, G. Berselli, S. Pirozzi, C. Natale, G. De Maria, and C. May, “Innovative technologies for the next generation of robotic hands,” in Advanced Bimanual Manipulation (Springer Tracts in Advanced Robotics (STAR)). New York: Springer-Verlag, 2012, pp. 173–218.[28] G. Berselli, M. Piccinini, G. Palli, and G. Vassura, “Engineering design of f luid-filled soft covers for robotic contact interfaces: Guidelines, nonlinear modeling, and experimental validation,” IEEE Trans. Robot., vol. 27, no. 3, pp. 436–449, 2011.

Claudio Melchiorri, DEI, Universitá di Bologna, Italy. E-mail: [email protected].

Gianluca Palli, DEI, Universitá di Bologna, Italy. E-mail: [email protected].

Giovanni Berselli, DIEF, University of Modena and Reggio Emilia, Modena, Italy. E-mail: [email protected].

Gabriele Vassura, DIEM, Universitá di Bologna, Italy. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________________

__________________

______________

__________________











hen robot researchers talk about robot middle ware, inevitably Player will enter

the discussion. This piece of software and its related tools have become one of the most popular software tools in

robotics research. Its range of hardware support and the flexibility it offers users, as well as its ease of use and shallow learning curve, have ensured its success.

Over a decade ago, the Player robot middleware was released to the world. What at first was just a small project to ease the use of a laboratory robot became one of the world’s most popular robotics research tools, particularly outside of industrial robotics. The middleware has been downloaded

over 100,000 times, while the Stage simulator has over 70,000 downloads and the Gazebo simulator more than 35,000 (statistics obtained from the SourceForge download tracker). The project as a whole still averages over 1,000 downloads every month.

How did this small set of tools created by a few young researchers rise to become such a commonly used piece of software for developing both real and simulated robots? In this article, we take a look at the history of Player and Stage—where they came from and why. We discuss why Player became and has remained popular for so long while so many other robot software projects have come and gone. We also discuss the Player project as it stands today and where it may go in the future. Throughout this article, we will refer to events that are shown in Figure 1.



History of Player and Stage Software©

ISTO

CK

PH

OTO

.CO

M/M

HE

LDE

NB

ER

GH

W


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


By Geoffrey Biggs, Radu Bogdan Rusu, Toby Collett, Brian Gerkey, and Richard Vaughan






An Evolving DesignThe design of Player has evolved and changed consider-ably over the years. However, the core principles of the architecture remain the same. It can still perform its origi-nal task, acting as a device server, but it can also compete, featurewise, with the latest component-based architec-tures. In this section, we discuss the evolution of Player’s design from its humble origins as a side project to make one lab’s robots easier to use.

Origins of a Robot ArchitectureIn 1998, the Interaction Lab at the University of Southern California (USC) used Pioneer robots from ActivMedia (now known as MobileRobots). These robots ran on the Ayllu middleware [1]. Ayllu is a behavior-based architecture targeted at distributed multirobot systems. It builds on the design of single-robot behavior-based control architectures, in particular the Subsumption [2] style of controller. How-ever, programming robots using it required following its con-troller design. This was quite limiting for researchers who wanted to try different controller styles. It also required the use of its own C-like custom language, adding another hurdle for developers.

What became Player was developed both to replace this somewhat restrictive control architecture and to allow controllers written for the new simulator developed in the lab, named “Arena,” to be executed on real robots without changes.

At first, controllers written for Arena were compiled indi-rectly, but it soon gained a socket interface. This allowed a separation between the simulator and robot controllers. It also led to Arena’s developers writing large pieces of robot control-ler code for use in Arena, which they also wanted to try on the lab’s Pioneer robots. Rather than porting their controllers to Ayllu, they developed ArenaServer.

ArenaServer ran on a Pioneer robot and presented the same socket interface as the Arena simulator. It was tied to both Ayllu and ActivMedia software but allowed robot control code designed for Arena to run, unchanged, on the lab’s real robots.

This experience led the developers of Arena and Arena Server to design the generic robot interface of their dreams, based on some simple ideas and UNIX concepts of abstraction:

● be as unprescriptive as possible● present a socket interface to the robot● hide all the details of dealing with the hardware● be free and open● work well with a simulator.A new server for the Pioneer robots was developed on the

basis of these ideas. This time, however, a custom driver was written for interacting with the robot’s controller. The new server was named “Golem.” An interface between Golem and Arena was created, allowing controllers written for one to work with the other. Golem was soon renamed as Player and Arena as Stage.

Work Begins on What Will Become Player 2.0

Work Begins on Arena SimulatorProposals for a More UNIX-Like Approach to Robot Software

Socket Interface to ArenaArenaServer Written to Run Arena Controllers on Pioneer Robots

Basic Concepts of Golem Laid Out

Golem/Arena Interface CreatedArenaServer Obseleted

Golem/Arena Renamed Player/StagePlayer/Stage Announced to the WorldBlug Reports from Outside Start Arriving

First Outside ContributionProject Hosting Moved to SourceForge

“Most Valuable Player” published

Passthrough Driver Developed

Gazebo StartedRADISH Established

ORCA 2 Released

Japanese RT Middleware Project Begins

Orocos Development BeginsFirst Orocos Release

Orocos Proposed

“On Device Abstractions” Published RETF Discusses Player as a “Defacto Standard”

Property Bag Support

Golem Created to Replace Ayullu and ArenaServer

Dynamically-Sized Arrays Support

Pluggable Interfaces Support“Player 2.0: Toward a Practical Robot ProgrammingFramework”Published

Player 3.0 Released

Player Summer School on Cognitive RobotsFirst Direct Funding for Player.Stage

Microsoft Robotics Studio (MSRS) 1.0 Released

ORCA 2 Switches to ICE

OpenRTM-Aist0.4.0

ROS Begins Development ROS 1.0 Released

Korean National OPRos Project Begins

OpenRTM-Aist0.4.2

OpenRTM-Aist1.0

MSRS 2008 goes free

Player Gains Windows Support

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Figure 1. Timeline showing major points in the development of Player, contrasted with other events in robot software.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







The Early DaysThe first publication concerning Player appeared in late 2000: the reference manual for version 0.7.4a [3]. At this point, Player was a device server for Pioneer robots, but already many of the concepts that the architecture is still based on today were beginning to develop.

The early Player protocol, shown in Figure 2(a), used a “small set of simple messages” [3] between clients and the server, communicating over TCP. The presence of the protocol even at this point illustrates the emphasis that was placed on the two benefits it brings to robotics: lan-guage/operating system independence and location inde-pendence. The use of a network connection also granted concurrent access to any number of clients.

The protocol was both simple and Pioneer-centric. In UNIX style, clients first had to request access to a device (a feature that still exists today and can be found in various forms in modern middleware such as ROS and Orocos). A single letter was used to indicate the device to access, imi-tating the semantics of the UNIX “open” system call. Seven device types were available, reflecting the hardware of the Pioneer. Command messages were sent by clients, with each device expecting its own specific format of command message. To keep the implementation simple, sent com-

mands did not receive replies. Data was sent by the server at a fixed configurable rate, 10 Hz by default. As with com-mands, the format varied by device type, although there was no true data message as we know it in Player today. Configuration messages could be sent to alter parameters of the server or the motor control. Unlike subsequent ver-sions of Player, these did not receive a reply.

The concept of device interfaces can be traced back to this early version of Player. Although tied to Pioneer hard-ware and not well defined, the concept was beginning to appear. Some messages still survive today, such as the request to the motor device to enable the motors.

The original client library bears little resemblance to the client libraries now included with Player. It did not have sepa-rate device proxies; all interaction was through an instance of the CRobot class, effectively a robot proxy. This class stored data received from the robot. Command values were written into data members of this class before calling a function that sent all command messages for all subscribed devices. Even at this early stage, though, Player had client libraries in multiple languages. C++, Tcl, and Java were provided, with Python under development.

The C++ client library was written as an example; the original intent was for the socket protocol to be the defining

c p Size

char char uint16(a)

(b)

(c)

Header

Header

Body

Body

uint16

uint16 uint16 uint16 uint16

uint16 uint16

uint16 uint16

uint32 uint32

uint32 uint32 uint32


uint32 uint8

uint8

uint8 uint8 Double

double double double

uint8


uint16

VelocityAngularVelocity

Stx

“Host” “Robot” Interface

x pos y pos Yaw x Speed

x Speed

y Speed

y Speed

Yaw Speed

Yaw Speed

State

State

Type

Type

TypeSub-type

Device Index

Index

Time (s) Time (us) Reserved Body SizeTimestamp

(s)

Time Stamp Sequence Body Size

Timestamp(us)

Figure 2. Changes in the Player message structure since the first version. (a) The command to move the robot in Player 0.7.4a. (b) The command in Player 1.65. Note the presence of multiple commands in the message body, with a “type” field to indicate which is being used. (c) The command in Player 2 and beyond. The message header was expanded to allow more flexibility, which in turn simplified message bodies.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







interface and that developers would be happy managing their own socket themselves. Over time, it was realized that the cli-ent API was just as important to end users.

One of the most important features of Player has always been that it makes no assumptions about the structure of robot control programs. This was a conscious design deci-sion from the beginning. It is perhaps the feature that most contributed to its early popularity. Having struggled with the limitations imposed by Ayllu, the Player developers decided that an open approach using a library-style API, rather than structured controllers, would give researchers more flexibility. Player therefore took a minimalist approach when compared with other architectures of the time, and even most architectures of today. In Player, the researcher is free to create controllers of any complexity and style they choose, from highly concurrent controllers to simple read–think–act loops.

The design of the Player architecture, shown in Figure 3,was asynchronous, allowing clients and devices to operate at their own speed. The server at this point used a large number of threads: two for each client (one reader, one writer) and one for each active device, as well as the thread listening for new clients.

Communication between clients and devices was via buffers in the server. Each device had two buffers, which stored the latest command and data. If new data or a new command arrived before the old was read, it overwrote the previous contents. This design decoupled devices from cli-ents, allowing all entities to operate at their own rate. Inter-nally, devices communicated by global shared memory.

The design did have its drawbacks. The fixed rate meant that clients could not determine whether the data it had just received was new or a repeat of old data (a “pull”

mode was later added to give clients control over receiv-ing data). The laser scanner operated at 5 Hz, for example, meaning laser data was out of date half the time. However, Player was intentionally designed this way for simplicity; it allowed clients to perform blocking reads.

By 2001, the internal server design had changed little. The authors of [4] formally defined Player as the protocol, not the server, and introduced the motivation for Player’s use of a socket abstraction:

● Distribution—Clients and servers could exist anywhere and in any number.

● Independence—Clients could be written in any lan-guage on any operating system supporting sockets.

● Convenience—Clients just had to connect and ask for access to the devices they were interested in to start receiving data.

The concept of virtual devices was first explicitly men-tioned in 2001. A device could be pure software, taking data from a hardware device and transforming it, before presenting it to clients. This concept allowed algorithms to be widely shared.

In 2001, Player was beginning to grow in popularity. Com-pared with the state of the art at the time, Player took a radi-cally different approach. Architectures focused on creating a development environment that suited a particular control phi-losophy. For example, Ayllu enforced a concurrent behavior-based control structure [1]. COLBERT/Saphira similarly focused on behavior-based control, using fuzzy blending [5].Architectures at this time often restricted the programmer to a specific language designed for the architecture, such as Ayllu’s C-like language. It was also still common for research robots to come with an unmodifiable library for control, with research-ers working within the constraints of this library.

PhysicalDevice D0

Device D0Thread

Device D0Command

Buffer

Client C0Reader Thread

Client C0Writer Thread

Client CxReader Thread

Client CxWriter Thread

Client C0

TCPConnection

Client Cx

Device D0DataBuffer

Device DnCommand

Buffer

Device DnDataBuffer

Shared Global Address Space

Device DnThread

Device-SpecificConnection(Serial, Etc.)

PhysicalDevice Dn

Current

Command

Data from

All Devices

Data from

All Devices

Current

Command

CurrentData

Commands toAll Devices

Commands toAll Devices

CurrentData

Figure 3. Architecture of Player for versions prior to 2.0.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







Player 1As Player moved through version 1, it gained an important feature: interfaces. The first clear statement on the separation of protocol, interfaces, and implementation was made in a publication in 2003 [6], which described the application of three well-known abstractions to robotics:

● Character devices—Based on the UNIX concept that everything is a file. Robot devices provide and consume streams of data.

● Interface/driver—interfaces define the contents of the character streams. Drivers implement one or more interfaces.

● Client/server—Provides location, language and con-troller design independence, and allows concurrent access to devices.

Through its abstractions, Player concentrated the “essen-tial qualities” of robot devices.

The abstract interfaces grew out of extending Player’s hardware support from just Pioneer robots to also include robots from RWI. The new drivers reused some of the structures from the Pioneer drivers. This indicated the over-lap between robot device interfaces and showed that, with suitable abstractions, a robot controller could run unchanged on different robot platforms (RWI support ini-tially had its own client-side support).

The authors of [6] also formalized the Player Abstract Device Interface (PADI), which by this point had grown to 29 interfaces. The Player protocol was now a well-defined flexible message-based protocol [shown in Figure 2(b)] and has remained message-based to the present day.

Many parts of the architecture were unchanged from ear-lier versions. The single data and command buffers per device, for example, were still intact. Drivers had been modu-

larized, allowing unneeded drivers to be left out when com-piling Player to reduce the binary size. This is especially important for embedded systems.

Virtual drivers were beginning to appear in greater num-bers. Fiducial detectors and Monte Carlo localization drivers were some of the first [7].

By version 1.3, in 2003, client libraries existed for C, C++, Tcl, Python, Java, and Common Lisp, and separate device proxies had been added.

Player 2Player 2.0, developed in 2004, was a major overhaul. It moved Player from a client/server-based model to a more flexible publisher/subscriber model, simplifying the architecture from the complex threads and buffers of Player 1 [8].

The new design, illustrated in Figure 4, attempted to meet several stated goals, as laid out in [9], including enhanced scalability, development process simplification, and transport independence. It had to do this while still maintaining compatibility with the by-now extensive code base developed by Player’s large community of users.

The motivations for the overhaul were many:● Player was being used in an increasingly wide range of

scenarios.● There was an increasing number of virtual drivers,

which Player was not originally designed to support.● The community was demanding more flexibility.● The number of contributors was growing, yet the API

(particularly for developing drivers) was difficult to use and data marshaling was not robust.

● The client/server model was restricting distributivity support.

● The single data and command message types per inter-face were limiting.

● Only one transport was available, and it was not always the best choice.

The overhaul redefined drivers as software components that could produce and consume messages.

To meet the goals, several changes were made to the architecture:

● The client/server model was replaced with a publisher/subscriber model. Client/server was preventing more general arrangements of servers and clients distributed across a network. In particular, the ability for servers to talk to each other was added.

● The transport was split out of the core, with a well-defined API, allowing multiple transports to be imple-mented (although only TCP and UDP were provided).

● Data marshaling was made the responsibility of the transport, not the sender or receiver, and implemented using automatically generated XDR functions.

● The message format was improved, allowing multiple command and data messages per interface. Interface design was greatly simplified by this new feature.

● The fixed data rate was removed, allowing devices to send data as fast as they liked.

Server

Main Thread

NonthreadedDriver

Buffer

ThreadedDriver

Buffer

ClientThread

Transport

Client

Buffer

Figure 4. Player 2’s message-queue-based architecture.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







● The driver API was simplified, using a callback model.The internal operation of the server was reworked to use

generic message queues rather than the single data and com-mand buffers per device. Each device and each client got its own message queue from which it could read at its own pace. This was an important change. It increased the flexibility of the server. Drivers could talk to each other by publishing in each other’s queues. This could even occur between servers, due to the expanded address format.

The use of XDR for marshaling was first proposed in [6]and saw its first appearance in Player 2.0. This removed a common source of errors when developing with Player, both for driver and client writers.

At the time of Player 2.0’s release, many more architectures were being developed and released using a variety of approaches. IPC and the Task Description Language extended C++ for task management [10]. CARMEN, based on IPC, provided a navigation toolkit [11]. CLARAty [12]used a two-tiered design, while CoolBot [13] and ROCI [14]provided component-based models.

Although the above architectures, like Player, developed nearly everything themselves, others began using of-the-shelf software. Orocos [15], MIRO [16], and OpenRTM-aist [17]built on top of CORBA. ORCA2 [18] used ICE.

The trend at this time was toward more and more frame-works being developed and released. This trend has arguably not subsided today. MARIE [19] appeared at about this time, seeking to unite the frameworks.

Player 3After the release of Player 2, development of Player began slowing down. Newer architectures were appearing, and the

developers were beginning to move on to other interests. Despite this, Player gained important new features during this time and continued on to a version 3 release in 2009. New features added in Player 3 included the following:

● Pluggable interfaces, allowing users to create new inter-faces and compile them as shared libraries.

● Support for dynamically sized arrays in the PADI. This was a major improvement that finally freed it of device-created limitations on data sizes, the most common of which being the regular need to increase the maximum size of a camera frame to support higher resolution cameras.

● Property bag support was added to provide a more device-specific form of support for the ioctl-style calls described in [6]. Property bags allowed individual devices to expose settings to subscribers such as baud rates that are not contained in an interface.

● Windows support for the server (at last).When Player 3 was released, it featured 44 interfaces.

Sixty-five interfaces have existed in total since Player began, with some being deprecated and removed over time. Player currently provides over 150 different drivers covering the full range of robot hardware and software.

Behind the ScenesAlthough we have described the evolution of Player’s design, it is worth considering the environment in which the design originated and evolved. (Much of this history can also be found on the Player Web site, at http://playerstage.source-forge.net/wiki/PlayerHistory.)

The original creators of Player are Brian Gerkey, Andrew Howard, Kasper Stoy, and Richard Vaughan, all members of

Figure 5. Known locations of Player users; there are likely to be many more. Player has been used in major universities and research institutes all around the world.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________________

__________________

http://www.qmags.com/clickthrough.asp?url=http://playerstage.sourceforge.net/wiki/PlayerHistory&id=18410&adid=P87E1






the Interaction Lab at USC. This lab was mainly focused on multirobot systems.

It was Gerkey and Vaughan who, in 1998, grew frustrated with Ayllu’s restrictions and developed the ideas that would become Arena and Golem, such as using UNIX concepts of abstraction. Golem was designed by Gerkey, Stoy, and Vaughan, on the basis of the experience of ArenaServer, and implemented by Gerkey and Stoy, while Vaughan concen-trated on Arena. At this point, the basic functionality of both Player and Stage were in place.

Andrew Howard arrived in 2000 and joined Gerkey and Vaughan in designing the interface/driver interaction frame-work that still underlies Player today. By this point, the major-ity of the Interaction Lab and Robotic Embedded Systems Lab robots at USC were running Player, and students were regularly using Stage in their work. Player/Stage was announced on the Interaction Lab’s Web site, and bug reports started arriving from people outside the lab.

In 2001, Player/Stage was moved to its present home at SourceForge (see http://playerstage.sourceforge.net), making it independent of the lab, and the first contribution from a third party, the University of Massachusetts, arrived. It was in heavy use at the USC by now. In 2002, Player/Stage received direct funding, and a three-dimensional (3-D) simulator, Gazebo, was started by Andrew Howard and Nate Koenig. By late 2002, Player was in use in over a dozen institutes [7] and was being used in both graduate and undergraduate classes. By mid-2003, this had grown to over 20 institutions [6].

Toby Collett, who was responsible for much of the archi-tecture overhaul in Player 2.0, joined the Robotics Research Group at the University of Auckland in 2003. After growing frustrated with the by-then antiquated system software for the group’s RWI B21r robot (the final straw was the discovery that it would only function on Redhat 6.2, which had a too old ver-sion of GCC for other software desired), Collett began looking for replacements. Player already had a driver for the B21r, and so Collett chose to use Player to drive the robot (see “Player in Education at the University of Auckland”).

However, problems soon surfaced in the design of the Player protocol when Collett realized that the B21r’s sensor geometry would change every time the robot turned. Over-coming this was not a major challenge, but to make things easier in the future, Collett undertook a major overhaul of Player’s internal architecture in 2004. This led to the release of Player 2.0 and earned him a place as a core Player developer. Biggs, from the same lab, soon joined him, eventually becom-ing the lead developer for the release of version 3. Collett and Biggs passed the torch on to Rich Mattes, from Penn State University, in 2010.

Making an Architecture a SuccessSince its first release, Player has been used all over the world (see Figure 5), with a large number of downloads (see Fig-ure 6). Initially, the factors that led to Player’s popularity were flexibility, community, and Stage.

Player allowed greater flexibility in controller design. Its contemporaries promoted “One True Way of Coding,” forcing

3,000

2,500

2,000

Dow

nloa

ds

1,500

1,000

500 v1.3

Monthly Downloads of Player

v3.0

v2.0

v2.1

v1.6v1.5

0

20012002

20032004

20052006

20072008

20092010

20112012

Summer School

Month

Figure 6. The number of downloads of Player from SourceForge.(a)

(b)

Figure 7. (a) Stage simulator as it appeared in 2000, when it was still named “Arena” (from [20]). (b) Stage as it appears today, showing its 2.5-D capabilities.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=http://playerstage.sourceforge.net&id=18410&adid=P88E1






the programmer into a particular controller structure. Player allowed the programmer to choose their own structure. Player also supported a variety of languages, while its contemporaries were typically limited to one, often custom-designed, lan-guage. Over the years, Player has supported C, C++, Java, Tcl, Python, Common Lisp, Matlab, and more.

It is recognized among open-source projects that a strong community is the key to a successful project. Player’s develop-ers cultivated a community around the project. Brian Gerkey was notable on the mailing lists for quickly answering any questions. The project as a whole was very open, with submit-ted patches quickly accepted into the source, irrespective of their origin, coding style, or even if they worked perfectly (“any code is good code”). The European summer school on Player was arguably a major driver in Player’s growth in Europe (see also “Player’s Role for European Academics”). Player’s strong community, which continues to this day, has allowed it to survive three generations of developers.

Stage was undoubtedly a major factor in Player’s initial and continuing success. By providing integrated support for a robot simulator, Player greatly increased its applicability and usefulness. The appearance of the 3-D Gazebo simulator in 2002 reinforced this. Stage, often assumed to be an offshoot of Player, actually predates it, as the Arena simulator [see Figure 7(a)] and was one of the inspirations for Player’s develop-ment. While the models in stage began by closely emulating Pioneer robots, as Player’s interfaces became more generic, so too did Stage’s models. They proved to be a useful test of new interface designs, as implementing the complete interface in a Stage model revealed ambiguities and contradictions [6]. This helped improve Player for its users. Stage has changed much

over the years. It is now a very powerful 2.5-D simulator. Figure 7(b) illustrates Stage as it is now.

The wide range of hardware support in Player is another factor in its success. When looking for software for a robot, one that already works is more likely to be chosen. When users then added additional drivers for their extra hardware, Player’s support grew more, further attracting new users.

Not every aspect of Player was a success. New technical fea-tures were sometimes ignored by the user community. For example, both the driver properties API and driver capabilities API were largely ignored. This may, in part, have been due to insufficient publicity about the features. Player also attempted to remove the choice between “push” and “pull” data modes, only to add it back at the last minute prior to Player 2’s release, upon finding that it introduced a serious limitation.

The Future of PlayerPlayer has been through two generations of main develop-ers and a major design overhaul since it began, and devel-opment of major new features is slowing. The Player 3 release did not feature any major new architectural

Player in Education at the University of Auckland

By Bruce MacDonaldWe have used Player in our robotics laboratory since 2004. We found from the start that Player was quite accessible to graduate students, reducing the time to come up to speed in programming any particular robot, and made it much easier for a student to take over a project from someone who had finished his/her study. Once the graduate students were comfortable with Player, we introduced it in the Robotics and Intelligent Systems course of the final undergraduate year. We ask these students to create a simple client for a following, foraging, manipulation, or navigation task. Students initially use Stage to test their clients. We use seven Pioneers, each with a simple arm, so that students can have some practical experience programming real robots. As a result, they are prepared to take up graduate study without any delay in learning how to program our robots. About 70–80 students take the course each year, with backgrounds in several different engineering disciplines. The organization of Player has encouraged students to improve the code base and to contribute ideas back. One group in their final year project improved the 2.5-D version of Stage, which illustrates how accessible the project is. Player has enabled a strong educational program in robotics, with an excellent progression to research projects.

Player’s Role for European Academics By Radu Bogdan Rusu

We started using Player around 2004 for student projects, performing experiments in Stage with multiagent system architectures. Its dedicated team of developers and their insatiable need to help newcomers made it very appealing for a lot of users to transition to becoming contributors, as they found the Player project compatible with their own needs. There was simply nothing of that kind in robotics at that point: a friendly talented team of international researchers and engineers working together on making robots useful, without any boundaries on how the project’s capabilities could be extended or what the project could be useful for.

Some of the first contributions from Europe came in the form of Player client libraries, with the Java client being one of the stronger contributions. Later on, advances in heterogeneous sensor networks for robotics applications and better 3-D sensing devices led to important contributions to Player in the form of new interfaces and drivers [23].

The Player Summer School took place in Munich, Germany, at the Technische Universitaet Muenchen (TUM), in the autumn of 2007. Being co-organized by TUM and SRI International, and with sponsorship from the European Robotics Network (EURON), the event was a major success. It brought together more than 50 students from all around the world for a seven-day marathon through Player for Cognitive Robotics Research. We had the pleasure of being able to assemble the entire Player core development team in one place, together with a list of top worldwide scientists and researchers, and cover topics from navigation to higher level reasoning, with a stop at Munich’s famous Hofbrauhaus brewery in the middle.

It was after the 2007 summer school that Player began getting an amazing response from the European community, with many contributions and fresh ideas flowing in.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







changes. However, it continues to be supported by its cur-rent maintainer, Rich Mattes, with regular bug fixes and enhancements to drivers and interfaces. New features are still added; in one recent example, a complete scripting interface was added to the server.

Although several new middleware projects have appeared that follow a much more distributed component model, Player is still capable of providing the same concepts. Drivers can be treated as components, and Player servers provide the communications middleware and deployment control.

While it may seem that Player’s popularity is waning, it still fills an important niche in robot software and is still a popular piece of software. Player 3, released in September 2009, has over 15,000 downloads as of mid-2011. A recent survey by Robotics Business Review showed that Player is still one of the most popular tools in robotics and is likely to continue to be so for some years [21]. The mailing list remains active, regu-larly receiving mail from new users requesting assistance.

Player’s future most likely lies in education. Its simple pro-gramming model is well suited to beginner and young pro-grammers working with simple robots. Encouraging an interest is important to programming education, and if LEGO Mindstorms [22] has shown us anything, it is that robots make programming interesting.

ConclusionsThe first paper published about Player ended with the follow-ing statement:

The acid test of Player will be its uptake. We hope that Player will develop over the next few years into a well-used tool. It will not suit every application, but it has proved useful in a variety of roles in our labs. [4]

We believe that Player has undoubtedly passed this acid test.From humble beginnings as an attempt to make a lab’s

robots more usable, Player has grown to become the one of the most popular robot software systems outside of industrial robots. Player solved a problem in robot soft-ware that no other architecture of the time did.

Player’s success can be attributed to its flexibility, which gave researchers the freedom they needed; its ease of use, allowing even beginner programmers to use it; the Stage simulator; and most importantly, its community support. The project is still a popular tool in robotics research today.

References[1] B. B. Werger, “Ayllu: Distributed port-arbitrated behavior-based control,”in Distributed Autonomous Robotic Systems, vol. 4, L. E. Parker, G. Bekey, and J. Barhen, Eds. Knoxville, TN: Springer-Verlag, Oct. 2000, pp. 25–34.[2] R. Brooks, “A robust layered control system for a mobile robot,” IEEE J. Robot. Autom., vol. 2, no. 1, pp. 14–23, Mar. 1986.[3] B. Gerkey, K. Sty, and R. T. Vaughan, “Player robot server,” Inst. Robot. Intell. Syst., School Eng., Univ. Southern California, Los Angeles, Tech. Rep. IRIS-00-392, Nov. 2000.[4] B. P. Gerkey, R. T. Vaughan, K. Stoy, A. Howard, G. S. Sukhatme, and M. J.Mataric, “Most valuable player: A robot device server for distributed control,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Wailea, HI, Oct. 2001, pp. 1226–1231.[5] K. Konolige, COLBERT: A Language for Reactive Control in Sapphira (Lec-ture Notes Artificial Intelligence), vol. 1303. New York: Springer-Verlag, 1997,pp. 31–52.

[6] R. T. Vaughan, B. P. Gerkey, and A. Howard, “On device abstractions for portable, reusable robot code,” in Proc. 2003 IEEE/RSJ Int. Conf. Intelligent Robots Systems, Las Vegas, NV, Oct. 2003, vol. 3, pp. 2421–2427.[7] B. Gerkey, R. T. Vaughan, and A. Howard, “The player/stage project: Tools for multi-robot and distributed sensor systems,” in Proc. 11th Int. Conf. Advanced Robotics, Coimbra, Portugal, Jun. 2003, pp. 317–323.[8] T. Collett, B. MacDonald, and B. Gerkey, “Player 2.0: Toward a practical robot programming framework,” in Proc. Australasian Conf. Robotics Auto-mation, Dec. 2005, pp. 1–7.[9] Y. H. Kuo and B. MacDonald, “A distributed real-time software framework for robotic applications,” in Proc. IEEE Int. Conf. Robotics Automation, Barce-lona, Spain, Apr. 2005, pp. 1976–1981.[10] R. Simmons and D. Apfelbaum, “A task description language for robot control,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 1998, vol. 3, pp. 1931–1937.[11] M. Montemerlo, N. Roy, and S. Thrun, “Perspectives on standardization in mobile robot programming: The Carnegie Mellon navigation toolkit,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2003, vol. 3, pp. 2436–2441.[12] I. Nesnas, R. Volpe, T. Estlin, H. Das, R. Petras, and D. Mutz, “Toward developing reusable software components for robotic applications,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, Nov. 2001, vol. 4, pp. 2375–2383.[13] A. Dominguez-Brito, D. Hernandez-Sosa, J. Isern-Gonzalez, and J.Cabrera-Gamez, “Integrating robotics software,” in Proc. IEEE Int. Conf. Robotics Automation, Apr. 2004, pp. 3423–3428.[14] L. Chaimowicz, A. Cowley, V. Sabella, and C. Taylor, “ROCI: A distributed framework for multi-robot perception and control,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2003, pp. 266–271.[15] P. Soetens and H. Bruyninckx, “Realtime hybrid task-based control for robots and machine tools,” in Proc. IEEE Int. Conf. Robotics Automation, Bar-celona, Spain, Apr. 2005, pp. 260–265.[16] H. Utz, S. Sablatnog, S. Enderle, and G. Kraetzschmar, “Miro - middle-ware for mobile robot applications,” IEEE Trans. Robot. Autom., vol. 18, no. 4,pp. 493–497, Aug. 2002.[17] N. Ando, T. Suehiro, K. Kitagaki, T. Kotoku, and W.-K. Yoon, “RT-mid-dleware: Distributed component middleware for RT (robot technology),” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2005, pp. 3933–3938.[18] A. Brooks, T. Kaupp, A. Makarenko, S. Williams, and A. Oreback, “Towards component-based robotics,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2005, pp. 163–168.[19] C. Cote, D. Letourneau, F. Michaud, J.-M. Valin, Y. Brosseau, C. Raievsky, M. Lemay, and V. Tran, “Code reusability tools for programming mobile robots,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, 2004, vol. 2, pp. 1820–1825.[20] R. T. Vaughan, K. Støy, G. S. Sukhatme, and M. J. Matarić, “Whistling in the dark: Cooperative trail following in uncertain localization space,” in Proc. 4th Int. Conf. Autonomous Agents, Barcelona, Spain, Jun. 2000, pp. 187–194.[21] Tools, Standards, and Platforms for Commercial Robotics Development: An Adoption Profile. (2010) [Online]. Available: http://www.roboticsbusiness-review.com/articles/newsletter view/tools-standards-and-platforms-for-com-mercial-robotics-development-an-adopti/[22] MINDSTORMS Robotics Invention System 2.0. (2006) [Online]. Availa-ble: http://mindstorms.lego.com/eng/products/ris/index.Asp[23] R. B. Rusu, B. Gerkey, and M. Beetz, “Robots in the kitchen: Exploiting ubiquitous sensing and actuation,” Robot. Auton. Syst., vol. 56, no. 10,pp. 844–856, 2008.

Geoffrey Biggs, National Institute of Advanced Industrial Sci-ence and Technology, Tsukuba, Ibaraki, Japan. E-mail: [email protected].

Radu Bogdan Rusu, Open Perception, California. E-mail: [email protected].

Toby Collett, Auckland, New Zealand. E-mail: [email protected].

Brian Gerkey, Open Source Robotics Foundation, Mountain View, California. E-mail: [email protected].

Richard Vaughan, Simon Fraser University, Burnaby, Canada. E-mail: [email protected].


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________

_________________

__________

_________________

__________

___________________________________________________________

______________


http://www.qmags.com/clickthrough.asp?url=http://www.mindstorms.lego.com/eng/products/ris/index.Asp&id=18410&adid=P90E2

http://www.qmags.com/clickthrough.asp?url=http://www.roboticsbusinessreview.com/articles/newsletterview/tools-standards-and-platforms-for-commercial-robotics-development-an-adopti/&id=18410&adid=P90E1










Members share fascinating fi rst-person stories

of technological innovations. Come read and

contribute your story.

IEEE Global History Networkwww.ieeeghn.org


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.ieeeghn.org&id=18410&adid=P91A1






ICRA 2013—A Snapshot for the Students

By Laura Margheri

Ijust got back from the 2013 IEEE International Conference on Robotics and Automation (ICRA) held in Karlsruhe, Germany. The conference

was a huge success, with fellow IEEE Robotics and Automation Society (RAS) volunteers saying: “One of the greatest ICRAs I have ever seen, maybe the best” and “Great, really great.” These accolades were a testament to the con-ference management and planning team, the plenary speakers, the orga-nized sessions and forums, the exhibi-tions, and the various events held throughout the week.

There have been many changes to the conference, and this year, I observed that younger members are more actively participating in ICRA events. While students and recent grad-uates are always behind the scenes at conferences, many are now leading the exhibitions and presenting at the ses-sions and in the forum. Many impres-sive advancements and technological and scientific discoveries were pre-sented, and I am sure we will see more emerging at future ICRAs.

This was more than a robotics and automation conference; this was a real conference with robots. There were robots flying in the air during the open-ing ceremony; robots walking, crawling, interacting, and talking around the exhibitions; and a robot creating a spec-tacular laser show at the reception.

ICRA was a result of the endless work of volunteers who ensured

that everything ran perfectly, commu-nicated constantly with the attendees, and managed the educational and social events. Of course, the IEEE Robotics & Automation Society Student Activities Committee (SAC) was involved and worked with the ICRA volunteers to organize many stu-dent events. In the middle of the week, the conference included the Lunch with Leaders and the Graduates of the Last Decade (GOLD) Luncheon, which were both very well attended and greatly appreciated. The Lunch with Leaders brought students, RAS leaders, and the plenary speakers together. IEEE President Peter Staecker and RAS Administrative Committee Member Robin Murphy gave great opening talks, sharing their passionate involve-ment and inspiring ideas.

The SAC also met to discuss the current status of the committee and plan future actions. The meeting was useful for recruiting new volunteers to join the committee, encourage partici-pation as liaisons to RAS Student Branch Chapters and the Women in Engineering RAS Ad Hoc Committee, and develop a student presence within the Automation Ad Hoc Committee and the Special Interest Group on Humanitarian Technology. The SAC plans to continue the student review program, and more information on this will follow. Things are changing and growing within the SAC, just as in the robotics and automation field.

RAS is looking to improve and changes have been in the works for the past few months, with brainstorming of

leaders and volunteers who attended the long range planning committee’s strategic planning meeting held in San Francisco, California, in January 2013. The purpose of this meeting was to hold interactive discussions regarding critical quests identified by RAS leaders, who are focused on shaping the strategy of the Society over the next few years. Conducted in the form of a strategic planning meeting, the brainstorming sessions were really interesting and very useful for identifying priorities and planning future actions. We discussed the Society as it is today by analyzing the mission, vision, and core values. We also reviewed the trends of science and technology as they concern the RAS, the profession, and the public.

Students are considered to be very important by the leaders of our commu-nity; they know we represent the future. The SAC’s main priority is to improve students’ interaction and professional experience with leaders in the robotics and automation field. We aim to contin-uously emphasize the voices and opin-ions of the student members and provide them with opportunities to be proactive. This goal was clearly highlighted during the ICRA and will be at the upcoming RAS flagship conferences IEEE/Robotics Society of Japan (RSJ) International Conference on Intelligent Robots and Systems and the IEEE International Conference on Automation Science and Engineering as well.

This is a great time to be in robotics. Robotics and automation is recognized

Digital Object Identifier 10.1109/MRA.2013.2271588Date of publication: 11 September 2013 (continued on p. 102)

STUDENT’S CORNER


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







CALL FOR STUDENT ACTIVITIES COMITTEE (SAC) CHAIR AND CO-CHAIRS

DEADLINE: 31 OCTOBER 2013The new SAC Chair will serve 2014-2015 and two SAC Co-chairs for 2014. The SAC aims to support the student members of RAS through programs that provide an exclusive resource to career and industry contacts and knowledge. The SAC Chair and Co-Chairs will drive these initatives as well as serving as liasons to a number of RAS standing committees. The SAC Chair serves as a voting member of the RAS Administrative Committee.

If you wish to be considered, please send the following information to the Vice President for Member Activities, Stefano Stramigioli at [email protected] and SAC Chair, Laura Margheri at [email protected].

1. A letter indicating why you are interested in serving on the SAC (2 pages maximum) 2. A copy of your CV or resumé (2 pages maximum) 3. A letter of support from a faculty member at your college or university, who is a member of IEEE RAS.

By joining the SAC as representative or volunteer, you will have the opportunity to enlarge your vision and perspectives, so don’t miss it!


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


______________________

___________________







WOMEN IN ENGINEERING

94

Attracting and Retaining Young Japanese Women in Robotics

By Gentiane Venture

Japan is well known for having one of the lowest percentages of women pursuing careers in engi-neering (<10%), both in industry

and at the academic level. Although some efforts have been made to attract women, cultural traditions are still too strong to be tackled, and only a few girls enter universities to study engineering (~11% versus 65% in social science and literature). Despite efforts to increase it, this figure has been stable for the past ten years. For mechanical engineering, the percentage drops to less than 10%, making female students, faculties, and professionals almost marginal.

The Tokyo University of Agriculture and Technology, also known as Nodo-kai, is one of the very few national uni-versities in Tokyo, with a faculty of engineering active in robotics research.Nodokai is also one of the most popular faculties of engineering among young females in Japan (with >20% female stu-dents enrolled in engineering, twice the national average). Nodokai’s scalable size, green atmosphere of the campus, and location in a quiet residential area that is still easily accessible by public transportation, play an important role. However, the university and the engi-neering faculty are also working to attract and recruit female students and faculty. The university has several pro-grams intended to reach women from the junior high school level to the researcher and faculty level to promote scientific paths and careers. These pro-

grams include summer schools, experi-mental programs, lectures, café-style discussions, a mentoring program, research grants, and support after child-birth, to name a few. I consider myself lucky to work in this environment, which promotes diversity. I would like to introduce and highlight two of these programs with which I have the chance to be involved for many years.

The first program is actually two in one: a summer school and an experi-mental program (Figures 1–3). Every summer, there are two major events tar-geting young female students interested in science and engineering. One is orga-nized at the university level and the other at the engineering faculty level. They both offer motivated students an opportunity to visit research labs, hear lectures, and attend experimental classes designed especially for them, where they can manipulate research equipment. Both events are scheduled for a full day. They each attract ~100 participants, but participants often do not come alone. More than one-third of them come with a chaperone, either their mother or their father. It might seem odd to welcome chaperones to our programs, but in

Japan, statistics show that parents’ choices and advice are crucial to stu-dents when choosing a major and a uni-versity. In about two-thirds of the cases, the parents’ decision strongly guides the student’s final decision. I believe this fig-ure is even higher for young girls because sociocultural parameters inher-ently decide what is suitable or not for a girl to learn, unconsciously reducing her degree of freedom. Therefore, by wel-coming the parents to our program, we have a unique opportunity to work directly with the decision makers. I put a lot of effort into trying to convince par-ents that engineering and robotics are perfectly suitable and decent paths for their daughters. There are a variety of opportunities and possibilities. Robotics is great as we use it everywhere, every day. At a low level now, we will use it much more in the future. Here, there is a vision where their daughter can be an active part of the shaping of the future. It gives them validation to be confident and that it is worth trying extreme chal-lenges like preparing an experimental program, where students have to pro-gram a robot to complete a given task and think logically!


Date of publication: 11 September 2013 Figure 1. The gait pattern analysis during an experimental program.



M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM






95

The second program is a mentoring program. This pro-gram has two aspects: 1) for undergraduate students and 2) for early career researchers (at the post-doctoral and assistant professor level). A pool of volunteer graduate students has been assembled to offer mentoring for under-graduates. Students can choose a mentor depending on their future career path, major, or affinity. The men-tors offer counseling, discus-sions, tips, and experiences. For the researchers, a pool of faculty members has been assembled to answer mentees’ needs. Ideally, mentor and mentee pick each other. There is only one rule: the mentee and mentor should contact each other at least once a month to discuss the points the mentee wants clar-ified or problems for which she wants some advice. I have mentored a few young researchers in our department for three years now, and I found it very enjoyable. It does not take up much of my time; we sometimes meet but often exchange e-mails. We talk a lot about

career choices and students’ supervision. I hope to be good at listening and give some good pieces of advice. Of course, each person’s experience is personal and unique, but it is important to share with each other what we have learned from our mistakes. The mentees can be on the

fast track, and the mentors are always kept up to date with the advances or stagnation of the youngest Women in Engi-neering’s (WIE’s) conditions.

Promoting WIE is an important duty for me that I enjoy fulfilling. I have had the chance to choose freely and entirely my career path and am fully supported in each decision by my family, my superiors, and my peers. I hope that each young girl can be offered the same opportu-nity and, for that, I want to reassure them and their par-ents that there is nothing that a woman cannot do. In par-ticular, robotics is such a broad and emerging field that there is room for every tal-ented and motivated young woman. More than covering the fields of mechanical engi-neering, electrical engineer-

ing, and computer science, robotics also makes it possible to work closely in col-laboration with so many other fields, including medicine, psychology, design, and arts, that there is definitely a career path that fits each individual.

Figure 2. The lecture before the experimental program explaining the aim of the experiments.

Figure 3. The demo of the NAO robot before programming it to analyze its center of pressure (CoP) trajectory with a Nintendo Wii.

George M. Whitesides (Harvard Univer-sity), Peter Fratzl (Max Planck Institute), Brad Nelson (ETH Zürich), Fritz Voll-rath (University of Oxford), and Nor-man Packard (European Centre for Liv-ing Technology).

To promote this research field, the cochair of the TC participates on the Editorial Board of Soft Robotics, a peer-reviewed journal dedicated to the sci-ence and engineering of soft materials in mobile machines, with the first issue available in spring 2014. The cochair also coordinates Robosoft, coordination actions on activities related to soft robotics research on a European scale. Robosoft is sponsored by E.U. Frame-work Programme 7 and will be started in October 2013.

Future ChallengesIn the next ten years, many scientific and technological challenges will be addressed by the TC . The first chal-lenge is to meet the requirement of developing functional and intelligent materials with controllable mechanical properties and adaptive functions for sensing and actuating, which are capa-ble of being fabricated and assembled, mass producible, and safe. The second is obtaining a thorough understanding of the way soft bodies are used in ani-mals on the basis of the guidance from biology and other related disciplines. Third, the establishment of a simula-tion and modeling technique of soft bodies behavior, particularly for boost-ing collaboration among researchers

from various disciplines. Finally, the integration aspects that will make the research ready for practical applica-tions, such as biomedical or rescue sce-narios, must be explored.

To address the challenges effec-tively, the TC has formed several com-munication channels. The main one is a public homepage (softrobotics.org) and its associated mailing list, with 265 registered e-mail addresses as of June 2013. Since May 2013, the TC produced a bimonthly newsletter to organize and promote the latest achievement of its members. We always encourage new members to join us. Anyone can subscribe to the mailing list by visiting https://sympa.ethz.ch/sympa/info/softrobotics.

TC SPOTLIGHT (continued from p. 25)

SEPTEMBER 2013 IEEE ROBOTICS & AUTOMATION MAGAZINE


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________

_____________________

http://www.qmags.com/clickthrough.asp?url=www.softrobotics.org&id=18410&adid=P95E1





http://www.qmags.com/clickthrough.asp?url=https://sympa.ethz.ch/sympa/info/softrobotics&id=18410&adid=P95E2



http://www.qmags.com/clickthrough.asp?url=https://sympa.ethz.ch/sympa/info/softrobotics&id=18410&adid=P95E2

96

ON THE SHELF


Hacking Electronics: An Illustrated DIY Guide for Makers and HobbyistsSimon Monk, McGraw Hill, New York, 2013, 274 pages.

acking has many connotations. Some may be less reputable, but in Hacking Electronics,hacking means, “Just do it!”

(p. 1). This book explores using, adapt-ing, and developing knowledge as you build and teaches you how to create many useful items. Included in every chapter are vivid color illustrations with apt headings, such as “what you need,” along with short directions with tips that teach principles and practices. This practical book might help highly theo-retical roboticists who get bogged down in abstract models bridge the gap between theory and practice. This book further lives up to its “hacking” name by showing ways of evolving an object, tak-ing it from one context into another, and adding functions and new uses. For example, in Chapter 1, in the section on how to solder, you learn how to remove a fan from the old computer gathering dust on your shelf and enlist it to blow solder fumes away.

This book starts at the literal begin-ning of practical skill building, with fundamentals such as wire stripping and gathering your basic toolkit. Chapter 2 provides a vivid, descriptive review of electronics basics that even advanced roboticists will appreciate. In

Chapter 3, clear construction tech-niques (basic hacks) are taught with a practical project in mind, making a light meter while learning to convert resis-tance to voltage. The authors also focus on general topics, including transistors, with details on choosing a bipolar tran-sistor for more varied options.

Chapter 4 provides a detailed focus on light-emitting diodes, including how to use a stripboard to make your cre-ations more permanent. In Chapter 5, batteries and power are covered with topics you would expect: controlling voltage from a battery, boosting voltage, using solar cells, and minimizing power consumption. Hacking a cell phone battery is included as well. General principles are hidden within the many interesting projects that will become useful for roboticists when designing robot circuits for optimal use of battery power.

The Arduino has become a popular microcontroller because of its low cost, open-source hardware design, accessi-ble integrated development environ-ment (IDE), and features such as clips for motor drivers. Chapter 6 on Arduinos takes you through a few hardware- and software-specific appli-cations such as hacking a toy for con-trol and controlling a relay from a Web page. You can learn useful general skills, such as measuring voltage with an Arduino and using Arduino shields.

Chapter 7 deals with hacking mod-ules, a great shortcut that can be readily accomplished with an Arduino. This chapter covers pyroelectric (passive)

infrared (PIR) motion sensor modules, a number of different rangefinder mod-ules, and wireless remote modules. There are also different motor control-lers, such as the power metal–oxide–semiconductor field-effect transistor (MOSFET) and H-Bridge modules for dc motors and stepper motors. Software instructions and suggestions for varia-tions in programming are included. One of the many projects in this chapter involves making a simple robot rover using a radio frequency remote control. The author takes you step by step through the entire process.

Sensors are an important compo-nent of many robots. Chapter 8 explains how to build and modify a variety of sensors that can detect gases, colors, vibrations, temperature, acceleration, and magnetic fields. You will also learn how to use an accelerometer.

Chapter 9 addresses audio hacks to amplify and screen sound by converting a stereo signal to mono and describes how to make a universal serial bus music controller.

Typically, we are tempted to throw away an electronic device instead of trying to repair or repurpose it. How-ever, with your hacking skills, you may be surprised to learn that many devices are easy to fix, and with hacking, you can improve them. Chapter 10 describes how to take things apart and put them back together, or at the very least, salvage some great parts for robotic applications. Following the

Digital Object Identifier 10.1109/MRA.2013.2272205Date of publication: 11 September 2013 (continued on p. 103)

H

Do It Yourself Electronics


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM







2013 RAS ADCOME L E C T I O NV O T I N G I S U N D E R W A YTHE IEEE ROBOTICS AND AUTOMATION SOCIETY ELECTION VOTING HAS BEGUN FOR RAS MEMBERS TO ELECT SIX NEW MEMBERS TO THE SOCIETY’S ADMINISTRATIVE COMMITTEE, TO SERVE THREE-YEAR TERMS BEGINNING 1 JANUARY 2014.

IN THE MONTH OF SEPTEMBER, VOTING MEMBERS (GRADUATE STUDENTS AND HIGHER GRADE MEMBERS) WILL RECEIVE THE ADCOM ELECTION INFORMATION PACKAGES DELIVERED VIA E-MAIL OR POSTAL MAIL IF REQUESTED OR E-MAIL IS NOT AVAILABLE. THE PACKAGE INCLUDES A SLATE OF THE CANDIDATES, THEIR BIOGRAPHIES AND POSITION STATEMENTS. CANDIDATE INFORMATION IS ALSO POSTED ON WWW.IEEE-RAS.ORG.

THE CANDIDATES FOR THE SIX POSITIONS ARE:

Geographical Area 1Greg Dudek, McGill University, CanadaSeth Hutchinson, University of Illinois Urbana-Champaign, USA Peter Luh, University of Connecticut, USA Hong Zhang, University of Alberta, Canada

Geographical Area 2François Chaumette, Inria in Rennes, France Eugenio Guglielmelli, Universita Campus Bio-Medico, ItalyTamas Haidegger, Budapest University of Technology and Economics, HungaryBrad Nelson, ETH Zurich, Switzerland

Geographical Area 3Nak Chong, Japan Advanced Institute of Science & Technology, JapanToshio Fukuda, Nagoya University/Meijo University

/Beijing Institute of Technology, JapanHong Qiao, Chinese Academy of Sciences, China Dong Soo Kwon, KAIST, Korea Michael Yu Wang, The Chinese University of Hong Kong, China


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_______________

http://www.qmags.com/clickthrough.asp?url=www.ieee-ras.org&id=18410&adid=P97A1





98

SOCIETY NEWS

RAS Awards

Ruzena Bajcsy Presented with the 2013 IEEE Robotics and Automation Award

During the IEEE International Robotics and Automation Conference (ICRA) in Karl-sruhe, Germany, Ruzena

Bajcsy was presented with the 2013 IEEE Robotics and Automation Award “for contributions to computer vision, the active perception paradigm, and medical robotics” (Figure 1).

A driving force in the field of robot-ics for over three decades, Ruzena Bajc-sy’s pioneering work on machine vision and perception has helped robots to achieve humanlike performance.

During the 1980s, she was the first to recognize that active perception was needed to improve computer vision/information acquisition. Active percep-tion enables mobile robots to actively control camera positions and other image acquisition conditions. Bajcsy’s landmark work on computer vision also includes modeling of deformable objects, elastic model matching, and visual hyperacuity, which has had important implications for medical robotics and imaging. Bajcsy founded the General Robotics, Automation, Sensing, and Perception Laboratory in 1978 at the University of Pennsylvania. In 1998, she became the first woman to lead the National Science Foundation’s Directorate of Computer and Informa-tion Science and Engineering.

An IEEE Fellow, Bajcsy is the NEC Chair Professor with the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley.

The IEEE Robotics and Automa-tion Award was established in 2002 by the IEEE Board of Directors and is presented for contributions in the field of robotics and automation. It includes but is not limited to manu-facturing automation, robotics and automation in unstructured environ-ments, sensor design, integration and fusion, robot design, modeling, plan-

ning and control, methodologies for robotics and automation, and the quality of the nomination.

New IEEE Robotics and Automation Society Publication LeadersIEEE Robotics and Automation Maga-zine (RAM) is currently under the direction of Editor in Chief Eugenio Guglielmelli of the Campus Bio-Medico University, Roma (Italy) (Figure 2).

Previously, Guglielmelli served as an associate editor with RAM from 2008 to 2011 under Stefano Stramigioli and

Digital Object Identifier 10.1109/MRA.2013.2272206Date of publication: 11 September 2013 Figure 1. Ruzena Bajcsy with IEEE President Peter Staecker.



M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM






Y O U K N O W Y O U R S T U D E N T S N E E D I E E E I N F O R M A T I O N .

N O W T H E Y C A N H AV E I T. A N D Y O U C A N A F F O R D I T.

I E E E R E C O G N I Z E S T H E S P E C I A L N E E D S O F S M A L L E R C O L L E G E S , and wants students

to have access to the information that will put them on the path to career success. Now,

smaller colleges can subscribe to the same IEEE collections that large universities receive,

but at a lower price, based on your full-time enrollment and degree programs.

Find out more–visit www.ieee.org/learning


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.ieee.org/learning&id=18410&adid=P99A1






Peter Corke. His first major IEEE Robotics and Automation Society (RAS) position was as secretary in 2002–2003. He was a founding cochair of the Tech-nical Committee (TC) on Rehabilitation Robotics in 2003. He served as an asso-ciate vice president for Technical Activi-ties in 2008–2009, and he is currently serving as an associate vice president for Member Activities. He has been an asso-ciate editor for IEEE Transactions on Robotics (T-RO) since 2009 and was a guest editor of a special issue on Rehabil-itation Robotics in June 2009. In 2012, he made important contributions to our conferences as the general chair of IEEE RAS\EMBS BioRob and the program chair of IEEE/Robotics Society of Japan (RSJ) International Conference on Intel-ligent Robots and Systems (IROS) 2012. He is active in several European research projects and is known for his scholarly achievements in the field of biomedical robotics.

From October 1, Frank Park (Fig-ure 3) will serve as the editor-in-chief of T-RO. Park has considerable expe-rience as an editor of T-RO and before

that, in 2004–2008, as an associate edi-tor of the former IEEE Transactions on Robotics and Automation (T-RA). He has ably served as an editor on the Conference Editorial Board for IROS and is also one of the part editors of the Springer Handbook of Robotics. He has been very active with RAS, for which he has served as the secretary for two different terms. He is currently the cochair of the Awards Committee. He is a Fellow of the IEEE for his contribu-tions to geometric methods in robot

mechanics. As a scholar in this field, he has extensively published in T-RO and other journals. In addition to his strong scientific reputation, he is recognized for his reliability and personal effective-ness in any position in which he has served.

Special thanks go to the retiring editors-in-chief: Peter Corke, who served RAM for three and half years with dedication and creativity, and Seth Hutchinson, who served T-RObetween October 2008 and September 2013 with excellence and leadership. Their efforts and service to RAS are very much appreciated.

Automation in Health-Care Management—Now an RAS TCThe RAS TC on Automation in Health-Care Management was approved by the RAS Administrative Committee. The TC aims to foster the development of theoretical and algorithmic tools, advance tools toward the solution of real-world problems, and advance theo-retical understanding of adequate algo-rithmic foundation for the planning and management of complex health-care service systems that can be applied in common problems and methodolog-ical approaches. The TC is chaired by Walter Ukovich, University of Trieste, Italy; Houshang Darabi, University of Illinois, Chicago; Maria Pia Fanti, Poly-technic of Bari, Italy; and Gregory Far-aut, ENS-Cachan, France. For more details visit us at www.ieee-ras.org, under Technical Committees.

RAS Egypt Chapter Receives 2012 Region 8 Chapter of the Year AwardThe RAS Egypt Chapter was awarded the 2012 Region 8 Chapter of the Year for outstanding achievements and suc-cessful operations throughout the year (Figure 4). As the Chapter has been very active not only in the typical Chapter activities, such as holding meetings, invited talks, and so forth, but also has engaged in humanitarian activities of national importance, including tackling the landmine prob-lem in northwestern Egypt. Further-more, the interaction of the Chapter

with undergraduate students is note-worthy and will certainly reflect both on student member enrollment and current member satisfaction.

Through carrying out these activi-ties, the RAS Egypt Chapter has put into practice the mission of IEEE, “to foster technological innovation and excellence for the benefit of humanity.”

For more information on the Chap-ter, visit us at http://www.ras-egypt.org/.

Growing Around the GlobeRAS is continuing to add branches around the world. Visit www.ieee-ras.org to find out more about a branch near you, and the newly formed Sweden Section Chapter and the following Stu-dent Branch Chapters at the SCMS School of Engineering and Technology, International Institute of IT—Bhu-baneswar, Universidade de Brasilia, and the Florida Institute of Technology.

Chapter Events Does your chapter have an upcoming event or a photo from a recent event? Send the details to [email protected] to have the event included on the RAS Calen-dar at www.ieee-ras.org and included here in the “Society News.”

Latin American Robotics Symposium–Latin American Robotics Competition 2013 in PeruThe X Latin American Robotics Sympo-sium (LARS), the XII Latin American Robotics Competition (LARC), and the I School Robotics Olympiad will be held 21–27 October 2013 in the southern Peruvian city of Arequipa and is being organized with the RAS Peru Chapter.

Figure 4. Dr. Ahmed Darwish, IEEE Section chair presents the award to Alaa Khamis, Chapter chair.

Figure 2. Eugenio Guglielmelli.

Figure 3. Frank Park.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


________

__

http://www.qmags.com/clickthrough.asp?url=www.ieee-ras.org&id=18410&adid=P100E4


http://www.qmags.com/clickthrough.asp?url=http://www.ras-egypt.org/&id=18410&adid=P100E2







Learn more about IEEE Open Access Publishing:

www.ieee.org/open-access

What does IEEE Open Access mean to an author?


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


http://www.qmags.com/clickthrough.asp?url=www.ieee.org/open-access&id=18410&adid=P101A1






This event brings together researchers and students from Latin American countries and those around the world to present recent advances in the field of robotics and participate in robot-ics competitions. Previous LARS–LARC events show that these events offer plenty of technology, modernity, and friendship.

More information regarding the call for papers and competi-tions can be found at http://ucsp.edu.pe/lars2013.

New Zealand Chapter Offers Trends in Manufacturing Automation ProgramAlexander Verl, director of the Fraunhofer Institute of Manufacturing Engineering and Automation (IPA) and the University of Stuttgart’s Institute of Control Engineering for Machine Tools and Manufacturing Units (ISW), was invited by the IEEE RAS New Zealand Chapter to give a technical seminar. The seminar was hosted by XiaoQi Chen, chair of the IEEE RAS New Zealand Chapter, and was attended by 16 researchers and engineers.

Verl gave a comprehensive overview of trends in various areas of manufac-turing automation (Figure 5). He started with a brief history of IPA, which is staffed with 199 scientists, 296 research assistants, and 108 lab/administrative staff, with the mission to develop future technologies for Germany’s advanced manufacturing industries. He presented a broad range of research projects, such as machining with robots, assistive robots, digital factory, orthopedics and motion systems, biological and medical technologies, and high-altitude power plants, three-dimensional (3-D) print-ing technology, entertainment robots, and so forth. The seminar stimulated

vivid discussion, and the relevance of robotics research relevant to New Zealand industry was delib-erated.

RAS Member Appointed Minister for Education, Universities, and Research in ItalyMaria Chiara Carrozza, an active IEEE RAS member of the Italian Chapter, has been appointed Min-ister for Education, Universities, and Research in Italy’s newly

formed government. Carrozza is a robotics expert, professor of bioengi-neering, and has served as rector of the Scuola Superiore Sant’Anna in Pisa.

AFRON Recognized with Tribeca Disruptive Innovation AwardThe founders of the African Robotics Network, Ken Goldberg, University of California, Berkeley, and Ayorkor Kor-sah, Ashesi University, Ghana, were pre-sented with the 2013 Tribeca Disruptive Innovation Award. The award was given in recognition of the “10 Dollar Robot” Design Challenge, which was sponsored by RAS.

Figure 5. Alexander Verl addresses members of the New Zealand Chapter with trends in manufacturing automation.

as a major engineering area and as an interdisciplinary nucleus integrating other sciences such as medicine, neuro-science, biology, and more. This field has a strong impact on global well

being, and therefore we must be cre-ative, passionate, and visionary.

I encourage you to join the SAC. It will give you the opportunity to expand your outlook and provide an

exclusive resource for your career and continuing knowledge sharing. Do not miss it; get involved, be active, and get connected.

STUDENT’S CORNER (continued from p. 92)

INDUSTRIAL ACTIVITIES (continued from p. 21)

where progress was assumed to have ceased. The bionic leg not only pro-vides assistance while the device is being worn but by responding to the patient’s own intended movements, it allows them to actively participate in the sit–stand, over-ground walking, and stair-climbing exercises that are critical to the recovery of gait and balance.

As with any startup in the health-care space, funding the company was a continual challenge, especially during and after the 2008 worldwide financial crisis. We survived those years through the dedication of our team and their belief in the great social benefits of what they were doing. In April, just before we were awarded the 2013 IEEE/IFR Invention and Entrepreneurship Award,

Tibion was acquired by AlterG, the company that pioneered another revo-lutionary therapy system, the AlterG Anti-Gravity Treadmill, which is used to provide comfortable partial-weight-bearing treadmill training. With the added resources of AlterG, we plan to scale manufacturing, sales, and develop-ment to expand the availability of bionic leg therapy worldwide.


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


__________

http://www.qmags.com/clickthrough.asp?url=http://ucsp.edu.pe/lars2013&id=18410&adid=P102E1








CALENDAR


201317–20 SeptemberSICE 2013: Conference of the Society of Instrument and Control Engineers of Japan. Nagoya, Japan. http://www.sice.or.jp/sice2013

21–29 SeptemberSummer Screws 2013—International Summer School on Screw-Theory Based Methods in Robotics. Florianop-olis, Brazil.

25–27 SeptemberECMR 2013: European Conference on Mobile Robots. Barcelona, Spain. http://www.iri.upc.edu/ecmr13/

15–17 OctoberHumanoids 2013: IEEE-RAS Interna-tional Conference on Humanoid Robots. Atlanta, Georgia, USA. http://www.humanoids2013.com/

20–23 OctoberICCAS 2013: International Confer-ence on Control, Automation and Systems. Gwangju, Korea. http://2013.iccas.org/

21–26 OctoberSSRR 2013: IEEE International Sym-posium on Safety, Security, and Rescue Robotics. Linkoping, Sweden.

30 October–2 NovemberURAI 2013: International Conference on Ubiquitous Robots and Ambient Intelligence. Jeju, Korea. http://www.kros.org/urai2013/

3–7 NovemberIROS 2013: IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan. http://www.iros2013.org/

5–9 NovemberARSO 2013: IEEE Workshop on Advanced Robotics and Its Social Impacts. Tokyo, Japan. http://www.arso2012.org/ARSO_2013_CFP.pdf

25–29 NovemberICAR 2013: International Conference on Advanced Robotics. Montevideo, Uruguay. http://www.icar2013.org/

15–17 DecemberSII 2013-IEEE/SICE International Symposium on System Integration. Kobe, Japan. http://sii2013.net/

201426–30 JanuaryMEMS 2014: IEEE 27th International Conference on Micro Electro Mechan-ical Systems. San Francisco, California, USA. http://www.mems2014.org

31 May–5 JuneICRA 2014: IEEE International Con-ference on Robotics and Automation. Hong Kong, China. http://www.icra2014.com/

12–15 AugustBioRob 2014: IEEE RAS and EMBS International Conference on Biomed-ical Robotics and Biomechatronics.San Paulo, Brazil


crucial warning to always unplug any device before you work on it, this book provides helpful tips for disassembly and reassembly, along with how to check fuses and components, test bat-teries and heating elements, and scav-enge for parts.

The final chapter provides simple instructions for using a multimeter and briefly introduces the oscilloscope. Overall, Hacking Electronics is accessi-ble, clear, and helpful. Therefore, roll up your sleeves and just do it!

—Reviewed by C. Alexander Simpkins, Ph.D., and

Annellen M. Simpkins, Ph.D., San Diego, California.

ON THE SHELF (continued from p. 96)


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


____

_________________

___________

http://www.qmags.com/clickthrough.asp?url=http://www.icra2014.com/&id=18410&adid=P103E11

http://www.qmags.com/clickthrough.asp?url=http://www.mems2014.org&id=18410&adid=P103E10

http://www.qmags.com/clickthrough.asp?url=http://sii2013.net/&id=18410&adid=P103E9

http://www.qmags.com/clickthrough.asp?url=http://www.icar2013.org/&id=18410&adid=P103E8

http://www.qmags.com/clickthrough.asp?url=http://www.arso2012.org/ARSO_2013_CFP.pdf&id=18410&adid=P103E7

http://www.qmags.com/clickthrough.asp?url=http://www.iros2013.org/&id=18410&adid=P103E6

http://www.qmags.com/clickthrough.asp?url=http://www.kros.org/urai2013/&id=18410&adid=P103E5

http://www.qmags.com/clickthrough.asp?url=http://2013.iccas.org/&id=18410&adid=P103E4

http://www.qmags.com/clickthrough.asp?url=http://www.humanoids2013.com/&id=18410&adid=P103E3

http://www.qmags.com/clickthrough.asp?url=http://www.iri.upc.edu/ecmr13/&id=18410&adid=P103E2

http://www.qmags.com/clickthrough.asp?url=http://www.sice.or.jp/sice2013&id=18410&adid=P103E1








AD INDEX

AD SALES OFFICES

Advertiser Page URL PhoneAdept MobileRobots Cover 2 www.mobilerobots.com +1 603 881 7960ATI Industrial Automation 23 www.ati-ia.com/rasBarrett Technology, Inc. Cover 4 www.barrett.com/xwam +1 617 252 9000Butterfly Haptics, LLC 20 http://butterflyhaptics.comCyberbotics Ltd. 5 www.cyberbotics.comEH Publishing 19 www. robobusiness.comForce Dimension Cover 3 www.forcedimension.com +41 22 362 6570Forest City Gear 7 www.forestcitygear.com +1 815 623 2168IEEE Marketing Dept. 13 www.ieee.org/tryieeexploreJohn Wiley & Sons, Inc. 15 www.wiley.com/ieee +1 877 762 2974KUKA youBot 3 www.youbot-store.com +49 1805 968 268Lacquey 17 www.lacquey.nl/IEEEReflexxes GmbH 21 www.reflexxes.comRobotnik Automation 10 www.robotnik.euSchunk USA 11 www.us.schunk.com/machine-potentialServoCity 8-9 www.ServoCity.com +1 620 221 0123


The Advertisers Index contained in this issue is compiled as a service to our readers and advertisers: the publisher is not liable for errors or omissions although every efforts is made to ensure its accuracy. Be sure to let our advertisers know you found them through IEEE Robotics & Automation Magazine.

James A. VickSr. Director, Advertising BusinessPhone: +1 212 419 7767Fax: +1 212 419 [email protected]

Marion DelaneyAdvertising Sales DirectorPhone: +1 415 863 4717Fax: +1 415 863 [email protected]

Susan SchneidermanBusiness Development ManagerPhone: +1 732 562 3946Fax: +1 732 981 [email protected]

Product AdvertisingMidAtlanticLisa RinaldoPhone: +1 732 772 0160Fax: +1 732 772 [email protected], NJ, PA, DE, MD, DC, KY, WV

New England/South CentralEastern CanadaJody EstabrookPhone: +1 774 283 4528Fax: +1 774 283 [email protected], ME, VT, NH, MA, RI,AR, LA, OK, TXCanada: Quebec, Nova Scotia,Newfoundland,Prince Edward Island,New Brunswick

SoutheastThomas FlynnPhone: +1 770 645 2944Fax: +1 770 993 [email protected], NC, SC, GA, FL, AL, MS, TN

Midwest/Central CanadaDave JonesPhone: +1 708 442 5633Fax: +1 708 442 [email protected], IL, IA, KS, MN, MO, NE, ND, SD, WICanada: Manitoba, Saskatchewan,Alberta

Midwest/Ontario, CanadaWill HamiltonPhone: +1 269 381 2156Fax: +1 269 381 [email protected], MI. Canada: Ontario

West Coast/Mountain StatesWestern CanadaMarshall RubinPhone: +1 818 888 2407Fax: +1 818 888 [email protected], CO, HI, NM, NV, UT, AK, ID, MT, WY, OR, WA, CA.Canada: British Columbia

Europe/Africa/Middle East/Asia/Far East/Pacific RimLouise SmithPhone: +44 1875 825 700Fax: +44 1875 825 [email protected], Africa, Middle East, Asia, Far East, Pacific Rim, Australia, New Zealand

Recruitment AdvertisingMidatlanticLisa RinaldoPhone: +1 732 772 0160Fax: +1 732 772 [email protected], NJ, CT, PA, DE, MD, DC,KY, WV

New England/Eastern CanadaLiza ReichPhone: +1 212 419 7578Fax: +1 212 419 [email protected], VT, NH, MA, RICanada: Quebec, Nova Scotia,NewfoundlandPrince EdwardIsland, New Brunswick

SoutheastCathy FlynnPhone: +1 770 645 2944Fax: +1 770 993 [email protected], NC, SC, GA, FL, AL, MS, TN

Midwest/South CentralCentral CanadaDarcy GiovingoPhone: +1 847 498 4520Fax: +1 847 498 [email protected], IL, IN, IA, KS, LA,MI, MN, MO, NE, ND,SD, OH, OK, TX, WI.Canada: Ontario, Manitoba,Saskatchewan, Alberta

West Coast/Southwest/Mountain States/AsiaTim MattesonPhone: +1 310 836 4064Fax: +1 310 836 [email protected], CO, HI, NV, NM,UT, CA, AK, ID, MT,WY, OR, WA. Canada: BritishColumbia

Europe/Africa/Middle EastLouise SmithPhone: +44 1875 825 700Fax: +44 1875 825 [email protected], Africa, Middle East


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


___________

___________

________

___________

___________

___________________________________________

______________________

___________

_____________________ ___________

______________________

http://www.qmags.com/clickthrough.asp?url=www.ServoCity.com&id=18410&adid=P104E16

http://www.qmags.com/clickthrough.asp?url=www.us.schunk.com/machine-potential&id=18410&adid=P104E15

http://www.qmags.com/clickthrough.asp?url=www.robotnik.eu&id=18410&adid=P104E14

http://www.qmags.com/clickthrough.asp?url=www.reflexxes.com&id=18410&adid=P104E13

http://www.qmags.com/clickthrough.asp?url=www.lacquey.nl/IEEE&id=18410&adid=P104E12

http://www.qmags.com/clickthrough.asp?url=www.youbot-store.com&id=18410&adid=P104E11

http://www.qmags.com/clickthrough.asp?url=www.wiley.com/ieee&id=18410&adid=P104E10

http://www.qmags.com/clickthrough.asp?url=www.ieee.org/tryieeexplore&id=18410&adid=P104E9

http://www.qmags.com/clickthrough.asp?url=www.forestcitygear.com&id=18410&adid=P104E8

http://www.qmags.com/clickthrough.asp?url=www.forcedimension.com&id=18410&adid=P104E7

http://www.qmags.com/clickthrough.asp?url=www.robobusiness.com&id=18410&adid=P104E6

http://www.qmags.com/clickthrough.asp?url=www.cyberbotics.com&id=18410&adid=P104E5

http://www.qmags.com/clickthrough.asp?url=http://butterflyhaptics.com&id=18410&adid=P104E4

http://www.qmags.com/clickthrough.asp?url=www.barrett.com/xwam&id=18410&adid=P104E3

http://www.qmags.com/clickthrough.asp?url=www.ati-ia.com/ras&id=18410&adid=P104E2

http://www.qmags.com/clickthrough.asp?url=www.mobilerobots.com&id=18410&adid=P104E1





















To display the digital world to your hands | to invent new ways to interact with computers and machines | we manufacture and market the finest precision master haptic devices for leading-edge applications in research, medical and industry.

With its unique 7 active degrees-of-freedom, the sigma.7 is at the heart of the Force Dimension master console designed for seated operators. The two end-effectors cover the user’s natural range of motion and offer a very high level of forces and torques, making it the most accomplished and versatile master console available today. The combination of full gravity compensation and driftless calibration contributes to greater user comfort and accuracy. Conceived and manufactured in Switzerland, the Force Dimension master console with dual sigma.7 is customizable and designed for demanding applications where performance and reliability are critical.

Force DimensionSwitzerland

[email protected]


M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_________________

http://www.qmags.com/clickthrough.asp?url=www.forcedimension.com&id=18410&adid=PCOVER 3A1







M AGAZ INE

oboticsutomation


M AGAZ INE

oboticsutomation

qqM

Mq

qM

MqM


qqM

Mq

qM

MqM


_________

_________________________

_____________

http://www.qmags.com/clickthrough.asp?url=www.barrett.com&id=18410&adid=PCOVER 4A2

http://www.qmags.com/clickthrough.asp?url=http://www.barrett.com/xwam&id=18410&adid=PCOVER 4A1

http://www.qmags.com/clickthrough.asp?url=www.barrettcommunications.com&id=18410&adid=PCOVER 4A3





RAM_20130901_Sep_2013

Documents

Transcript of RAM_20130901_Sep_2013