An integrated robotic system for antipersonnel mines detection

Control Engineering Practice 10 (2002) 1283–1291

An integrated robotic system for antipersonnel mines detection

Eric Colona,*, Ping Honga, Jean-Claude Habumuremyia, Ioan Dorofteia, Yvan Baudoina,Hichem Shalib, Dragomir Milojevicc, J!er #ome Weemaelsd

aRobotics Centre, Royal Military Academy, Avenue de la Renaissance 30, B-1000 Brussels, BelgiumbDepartment of Electronics and Information Processing VUB-ETRO-IRIS, Vrije Universiteit Brussel, Pleinlaan 2, B-1050 Brussels, Belgium

cService des Syst"emes Logiques et Num!eriques, Universit!e Libre de Bruxelles 50, Av. F. Roosevelt, CP 165/57 B-1050 Brussels, BelgiumdSCMeRO, Universit!e Libre de Bruxelles 50, Av. F.Roosevelt, CP 165/42 B-1050 Brussels, Belgium

Received 14 January 2002; accepted 15 March 2002

Abstract

In this paper robotic systems for antipersonnel mines detection are presented. Robots with different locomotion principles have

been developed to carry mines detection sensors. In this project, many different aspects including mechanics, control and positioning

have been addressed. After a short review of the project and a description of the system components, the control and communication

architecture of the robotic detection systems is presented. One of the most important issues in this application was the

synchronisation of the processes between the microcontrollers and remote computers. The method that has been implemented has

proven to be robust and the different systems have been used to acquire very useful data for the research community. r 2002

Elsevier Science Ltd. All rights reserved.

Keywords: Robotics; Remote control; Microcontroller; Human machine interface; Sensors; Visual servoing

1. Introduction

The terrible antipersonnel landmines plague repre-sents a real challenge for the research community.Antipersonnel mines kill or mutilate tens of people everyday. Humanitarian deminers still use classical manualmethods because heavy demining vehicles cannotachieve a satisfying destruction percentage. This workis very slow, tedious, dangerous and costly. Further-more, the detection is not always reliable. Improvementscan be made by developing new sensors, by automatingthe detection sequence and by using different sensorssimultaneously (Baudoin & Colon, 1999). The RoyalMilitary Academy, leading the Belgian project Hudem,is focusing on the development of new data processingand fusion algorithms (Milisavljevic, 1999), on improve-ment of Ground Penetration Radar (GPR) (Scheers,1999) and on robotic systems that carry mines detectionsensors.

Among the different ways robots could help humandeminers, the scenarios described below are the most

realistic. Small autonomous vehicles equipped withdifferent sensors could run around an area to delimitthe surface that is really polluted with mines. This phasewhen done manually is the most dangerous one becausedeminers are working faster and are taking more risksthan during systematic detection. To study this firstaspect, a small wheeled electrical vehicle named Tridemhas been developed (Fig. 1). A first indoor prototypewas build to demonstrate the concept feasibility(Alexandre, Weemaaels, Doroftei, & Colon, 1998). Asecond version with more powerful motors and amodified frame has been developed for outdoor trials.

Once the actual mined area is delimited, a systematicscanning process can begin. It has been proved that theuse of different sensors could drastically improve thedetection efficiency and reliability. However, the datafusion process requires the registration of the dataacquired by the different sensors. This requirementjustifies the work done on the two other systems: the firstone which is a sliding pneumatic robots with a twodegrees of freedom scanner (Fig. 2) and the second onewhich is a modified explosive ordnance disposal vehiclethat carries a three-dimensional Cartesian scanningsystem (Fig. 3) (Colon, Baudoin, & Alexandre, 1998).The fact that the three systems have very different

*Corresponding author. Tel.: +32-2-737-6552; fax: +32-2-737-

6547.

E-mail address: [email protected] (E. Colon).

0967-0661/02/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.

PII: S 0 9 6 7 - 0 6 6 1 ( 0 2 ) 0 0 0 9 0 - 4

robotic architectures introduced some additional chal-lenges in the development of the control and interfacearchitecture. Also the different sensors used in theproject and more particularly their interface influencedthe system implementation.

The last aspect considered in this project is thedetermination of the robot’s location in the field. This isrequired for navigation but also for automatic produc-

tion of detection maps. For this purpose, a visualservoing system based on a pan-and-tilt colour camerahas been developed (Hong, Sahli, & Baudoin, 1998).This system tracks a colour beacon mounted on therobot and sends in real time the three-dimensionalposition of the sensor to the main control computer. Theprinciple is shown in Fig. 4.

Besides mobility trials, the systems have been used ondummy domestic minefields to record data for thesearchers working on sensor development and datafusion algorithms.

This article begins with an overview of the differentcomponents of the robotic acquisition systems. Next,the control and communication architecture is de-scribed. The emphasis is put on the control of the

Fig. 1. The Tridem with the metal detector.

Fig. 2. The pneumatic walking robot AMRU 4.

Fig. 3. The Hunter with a blue beacon.

Fig. 4. The tracking system principle.

E. Colon et al. / Control Engineering Practice 10 (2002) 1283–12911284

scanning and on the synchronisation between thedistributed processes. Finally, some results are presentedand future work is discussed.

2. System overview

Basically, the scanning robotic systems are composedof the following elements:

* the vehicles with their possible scanning device,* the mine detection sensors, and* the tracking and location system.

As described in the preceding section the roboticsystems can be divided into two categories: the oneshaving a scanning device that can be equipped withdifferent sensors and the wheeled robot that simplycarries a single sensor. In this case, the scanning of anarea can be obtained by moving the robot body itself.

The mechanical systems need to be controlled andtheir motion must be synchronised with the dataacquisition. The vehicle motion is also synchronisedwith the tracking and location process.

The vehicles and the scanners are described in thissection; the control and communication architecture ispresented in Section 3.

The first vehicle available in the project is calledHunter (Fig. 3); it is a small caterpillar vehicle that wasused by the Belgian Army for antiterrorism operations.The original manual control system of the vehicle hasbeen interfaced with a microcontroller such that thevehicle can now be controlled with a computer. Thescanner is actually a Cartesian robot with three degreesof freedom. A DC servomotor coupled with a planetarygear-head actuates each axis. An optical digital encoderis used for position and speed feedback and contactswitches have also been placed at the end of each axis.The useful area is 850� 500mm2 and the vertical axishas a travel distance of 500mm. The system can be usedon both sides of the vehicle and is powered withbatteries or with a power supply.

The second robot (AMRU4) is a pneumaticallyactuated walking machine (Fig. 2). Two sliding framesallow a linear motion and a rotating cylinder is used forchanging the motion direction. The upper part of therobot is a two degrees of freedom scanner that can carrydifferent kinds of mine detection sensors. As mentionedin the introduction, to develop efficient detectionmethods, precise and reliable data are essential. Thisfact justifies the development of those two systems,which are able to acquire multi-sensor data.

The third vehicle called Tridem (Fig. 1) has atriangular shape and stands on three wheels. The wheelsare connected to the frame as a three arms star. Theframe supports the electronics and the batteries. The

robot is equipped with three driving and three steeringmotors. The power is transmitted to the driving motorsby copper brushes. This solution gives the wheels acomplete rotation freedom around the steering axis.System modularity was one of the basic requirementsfor this robot. The wheels can be removed and replacedvery easily because all the wheels are identical and arefastened with fast screw connections. The electricalconnections are also very easy; the wires (control signaland power) are connected through standard DB9connectors which are embedded into the frame. It isan omnidirectional vehicle that has been designed forhighly unstructured grounds. This platform could beused to detect mines in areas which are dangerous forhuman deminers.

3. Control architecture

3.1. The components of the control architecture

The whole system has a multi-processing architectureand comprises the following components (Fig. 5):

* the HMI computer,* an embedded computer for data acquisition and

communication with the HMI computer,* the motion controller (microcontroller), and* the visual tracking and location computer.

The following functions are distributed among thedifferent processing systems:

* motion control,* processes synchronisation, and* data acquisition.

In this application, two kinds of interfaces are needed:communication interfaces (serial, Ethernet) and acquisi-tion interfaces (serial, GPIB, frame grabber).

The subsequent subsections give an overview of thedifferent components of the control architecture. Thelast subsection describes the integration and synchroni-sation of the different systems and processes.

3.2. The Master PC

The control architecture comprises two loops: a high-level control loop that runs on a PC and synchroniseshigh-level processes and a low-level control looprunning on a microcontroller which performs commoncontrol functions (position and speed control). TheMaster PC is responsible for the synchronisation ofthe robot motion with the positioning system and for thesynchronisation of the scanning with the data acquisi-tion. It sends configuration and trigger commands to the

E. Colon et al. / Control Engineering Practice 10 (2002) 1283–1291 1285

microcontroller through a serial link and performs thedata acquisition.

The role of the Master PC can be played by the HMIPC, when it is directly connected to the vehicles and tothe sensors, or by the embedded PC which commu-nicates via a TCP/IP link with the HMI PC. Thisremote acquisition can for the moment only beperformed with the metal detector (MD) (RS232 inter-face). The direct connection is required when acquiringdata from the GPR because it is an instrumentationsystem that cannot be easily displaced. It uses a normalPC HPIB interface card connected to a scope foracquiring data. When a portable sensor is available, itwill be possible to easily integrate it into the mobilesystem.

3.3. The motion and scanning controller

The three robotic systems use the same basicelectronics; the low-level control is realised with a 32-bit microcontroller that communicates through a seriallink with the Master PC. They can be remotelycontrolled through the same HMI, a program calledCoRoDe (Control of Robots for Demining). The HMImanages all control aspects: vehicle and scanningcontrol, communication and data acquisition. A highinteractivity and user-friendliness is reached throughthe use of efficient programming techniques (seeSection 3.7).

The 32-bit microcontroller is very versatile andpowerful; it performs different functions depending onthe system:

* it controls electrical motors (scanner, Tridem),* it monitors signals coming from contact switches,* it generates commands (replacing a PLC) for

controlling the valves on the pneumatic robot,* it generates analogue signals for the tracks of the

caterpillar vehicle, and* it hosts the main communication loop with the

Master PC.

The specific characteristics of the control methods aregiven in the remainder of this section.

What concerns the Hunter, two control levels can bedistinguished: a software only loop that managescommunication through the serial interface and themotion control functions which are interrupt based. Theserial port is polled and a switch-case instruction firesthe function corresponding to the received code. Mainhigh-level control functions for the Hunter are: setscanning limits, set position, set speeds, move to arealimits, start scanning y . During scanning, synchroni-sation signals are sent to the Master PC to trigger thedata acquisition. Two scanning modes are available: acontinuous mode and a discrete mode. In the discretemode, the scanner stops at every step waiting for the endof the data acquisition. The acquisition step can bechanged in the set-up dialog box of the HMI.

Fig. 5. The general system architecture.


In the moveTo function, the desired goal coordinatesare copied into shared variables that are read by themotion control functions. These functions are called byinterrupts at fixed intervals (a 4ms control period isused). The PI controller controls the motor speed byusing the time processing units (TPU) of the micro-controller which are configured to generate PWMsignals. A cascade negative feedback control combiningtwo loops has been implemented: the first loopcorresponds to a speed negative feedback control (theinner loop), while the other one is for position negativefeedback control (the outer loop).

The microcontroller also generates analogue signalsto drive the tracks of the Hunter. Two TPU’s channelsproduce two independent PWM signals for right and lefttracks. The PWM signals are filtered to generatecontinuous bipolar analog signals compatible with theold interface. As the driving motors do not haveencoders it actually acts as an open-loop controller.

In the case of the walking robot, the microcontrolleris essentially used as a software PLC. The control isimplemented like a finite-state machine and all outputsare binary signals. The TPUs are used to read theposition signal from incremental linear sensors placedalong the pneumatic cylinders. The main loop managesthe communication with the Master PC.

The control of the Tridem is similar to the one of theHunter but requires all TPUs of the controller. TheTPUs generate six PWM signals and capture the digitalencoders’ signals from the steering motors. The speed ofthe driving wheels are measured with tacho-generatorswhose signals are converted by the internal ADconverters. A classical PI algorithm is used for control-ling the wheels.

Different configurations are possible (Figs. 6a–c,respectively):

* Translate: all wheels are parallel.* Spin: the wheels are tangent to a circle.* Free: the normal motion mode in which the three

wheels are separately commanded to follow arbitrarytrajectories.

This latter mode is preferred for the detection becauseit keeps the MD pointed along the motion direction.

3.4. Sensors and acquisition interfaces

Three different sensors have been successfully used inthe project: an MD, a GPR and an infrared camera. Afourth one, a pyrometer, has been tested but abandoneddue to unsatisfactory results. The data-acquisitionprocess requires different interfaces: the MD has a serialinterface, the GPR data are read through the GPIBinterface of a high-speed oscilloscope, the imagescoming from the infrared camera are captured with aframe grabber (through a cable or a wireless connec-tion). We will see in Section 3.7 how the sensors’characteristics influence the scanning process and theway the control is realised.

3.5. Communication

A serial communication allows the transmission ofcommands between the Master PC and the microcon-troller (the transmission speed is 9600 baud). RadioEthernet links (protocol 802.11) are used to commu-nicate between the HMI PC and the embedded PC.

3.6. Location and tracking

A location system determines the robot’s positionswhich are used to automatically generate a map of thedetections (Hong, 2001). A pan-and-tilt colour cameratracks a coloured target mounted on the vehicle. Everyframe the ball object is extracted from the image (Fig. 7)and an estimation of its position is computed. As therobot moves, the camera follows the target in order tokeep it in the centre of the image. Furthermore, the sizeof the target is kept constant by using a motorised zoomand the value of the zoom gives an estimation of thedistance. As the orientation of the camera is also known,the ball is completely located in the field. In favourableconditions (uniform background, constant illumination)

Fig. 6. (a)–(c) Translate mode, spin mode and free mode


a precision of 30 cm at distances up to 15m is reached,which is enough for this application. In order to increasethe precision of the location process in varying condi-tions the coupling of the camera with a laser telemeter isnow considered.

3.7. Human machine interface

The graphical user interface of the control programCoRoDe is shown in Fig. 8. This program offers thefollowing functions:

* control of the vehicle,* configuration and control of the scanning system,* configuration of the sensors,* data visualisation,* data archiving, and* mapping.

The main area is devoted to data visualisation. It canalternatively display a map with successive locations ofthe robot or the sensors’ data.

* Displayed data can result from two different acquisi-tion methods:

* Global acquisition:* Infra-red images,* Video images (used to view and record pictures

from the scanned area).* Sequential acquisition:

* Single values organised as a 2D image (MD),* One-dimensional data merged into a 3D volume

(GPR).

The map contains a reference frame and consecutivepositions of the robot and scanned areas. The user canswitch between either configuration by clicking on abutton in the toolbar.

Data acquired during the scan process are saved intwo different formats: first as binary data for later

processing (double for GPR, double word for MD) andas 8-bits grey scale raw images for direct visualisation.The data acquisition, scanning, location computationand vehicle motion are integrated into a sequence that iscontrolled by the user with button commands lying in asingle toolbar. The interface is simple and intuitivethanks to the use of well-known symbols (VCR-like)and standard colours (see Fig. 8). In this application, itwas a requirement to let the user keep the control of theprocess; at every moment the user can pause, resume orstop the operations.

It is also essential to provide information duringinternal processing or timeouts. In this case, sensors’data are drawn on the screen as the scanning progresses.The position of the scanner relative to the maximumpositions, the status of the scanning sequence and themain options are also presented to the user and regularlyupdated. Finally, the use of additional communicationthreads (see next section) preserves the interaction withthe user interface. The next section provides implemen-tation details about the communication and thesynchronisation of processes.

3.8. Control process, communication and synchronisation

The robotic systems used for sequential data acquisi-tion have a more complex control architecture than thewheeled robot. The division of the control between themicrocontroller and the Master PC requires a perma-nent synchronisation of the processes. The mainprogram runs on a computer under the Windowsoperating system (OS) (Windows98 was used becauseat that time it was the only windows version compatiblewith the force-feedback joystick drivers, now Win-dows2000 or XP could be used as well). The commu-nication with the microcontroller runs in its own threadin order to keep the interactivity of the user interface.Each event coming from the microcontroller (begin orend of line, position trigger for data acquisition)through the serial port is intercepted by the commu-nication thread. For each event, corresponding messagesare posted to the view window (the program isimplemented with the MFC document-view scheme).Each message results in a call to the correspondingfunction of the window class (examples of such messagesare acquiredata, savedata, y). When using the discretescanning mode, this process is synchronous and themicrocontroller blocks until it receives the acknowl-edgement of the main process. The consequence is thatthe scanning is not continuous but stops at every step(typically every 2 cm). This was required by the MDbecause when triggered, this sensor has an acquisitionlatency of about 200ms which corresponds to 2 cm for aspeed of 10 cm/s and consequently to a 4 cm shiftbetween the forward and backward scanning lines. Thissynchronisation also removes the need to take into

Fig. 7. Colour target identification.


account the delays introduced by the OS messagequeuing mechanism and the acquisition duration. Thecommunication thread then enters in a blocking sectionand waits until the data acquisition has been performedand the lock released to send the acknowledged messageto the microcontroller.

Realising the communication between two threads bymessages allows to completely uncouple them (the otherway is to pass objects’ pointers to the control thread andto directly call functions from it). At each step, theposition of the scanner in the user interface is updated,data is acquired, plotted and saved.

However, this asynchronous control has two draw-backs: the scanning process is slower and it can producevibrations due to resonance phenomena. As mentionedearlier, the first limitation is also due to the sensors andcannot be avoided without changing the sensorsthemselves. Introducing random delays to avoid reso-nance frequencies can eliminate the second problem.Other possibilities for dealing with the acquisitionlatency is to calibrate the delays and to shift the dataevery two lines at the end of the acquisition or to alwaysscan in the same direction. The first solution wasrejected by the data processing team because itintroduces some extra pre-processing and can alter the

data. The second solution has as consequence to doublethe scanning duration and to cancel the benefit ofthe continuous scanning; it was consequently notconsidered.

What concerns the Tridem, there is no such synchro-nisation between the robot and the Master PC becausethe communication process is essentially asynchronous.The microcontroller receives configuration or motioncommands and executes them. An acknowledgement issent after the reception of the command, the executionis started and a new command is read from the serialport.

4. Results and future work

The two systems equipped with scanners have beensuccessfully used to test the GPR prototype and toacquire registered multi-sensors data (Fig. 9). Thesedata have been very valuable for the searchers workingon fusion methods. The wheeled robot has also beendemonstrated as a remotely controlled mine detectionsystem.

Different elements have contributed to the success ofthis project:

Fig. 8. The CoRoDe data visualisation window.


* the use of the same microcontroller for differentrobots,

* the use of object-oriented programming techniquesand language that allow to easily reuse buildingblocks in developing new applications or modifyingexisting ones,

* the hierarchical model of the control, and* the user-friendly user interface.

But despite these benefits, the application presentsseveral limitations:

* it is a monolithic application where control andinterface cannot be easily separated,

* it is OS specific,* adding a robot with new capabilities or a sensor with

new display requirements would demand importantmodifications of the program.

These drawbacks are not unique for this applicationbut are common characteristics of classic programs.

New ways for implementing the control and visualisa-tion software are consequently considered: components

and distributed components like CORBA or Java RMIare possible candidates. The requirements for the newarchitecture are listed below. It should:

* provide an universal user interface,* have plug and use capability,* be open to existing and future robots,* allow the concurrent control of different robots,* allow the easy reconfiguration of the application

(addition or removal of components: robots, sensors,users),

* allow several users to collaborate, and

* allow the users to incorporate their own controlalgorithms.

5. Conclusion

This paper has presented the results achieved by theRobotic Workgroup within the Belgian funded projectHudem. Robots using different locomotion techniqueshave been successfully used to acquire sensor data andto test and validate utilisation methods in differentscenarios. In this paper, the control and programmingarchitectures of these systems have been described. Theuse of object-oriented techniques for application devel-opment and the reuse of the same control hardware havecontributed to the success of this project. However, inthe next future new programming techniques willallow developers to write more open and reusableapplications.

At this stage, it has not been proven that roboticdetection of mines works better or faster than humandeminers. But the obtained results are encouraging andpave the way for an integrated solution that will someday help to solve this terrible plague.

Acknowledgements

This project would not have been possible without thecontribution of all the searchers involved in the Hudemproject and without the funding by the Belgian Ministryof Defence and the Secretary for Cooperation andDevelopment.

Fig. 9. Data acquisition using the prototype GPR and a metal detector.


References

Alexandre, P., Weemaaels, J., Doroftei, I., & Colon, E. (1998).

Development of a high mobility wheeled robot for humanitarian

mine clearance. Proceedings of the robotic and semi-robotic ground

vehicle technology. Orlando, USA: Aerosense-SPIE.

Baudoin, Y., & Colon, E. (1999). Humanitarian demining and

robotics: A difficult challenge. Proceedings of the Clawar99.

Portsmouth, UK.

Colon, E., Baudoin, Y., & Alexandre, P. (1998). Development of

mobile robots for mines detection. Proceedings of the Mechatro-

nics98. Skovde, Sweden.

Hong, P. (2001). Visual servoing for robot navigation: application in

humanitarian demining. Ph.D. thesis, Free University of Brussels,

Belgium.

Hong, P., Sahli, H., & Baudoin, Y. (1998). Color target detection and

tracking. Proceedings of the Clawar ’98. Brussels, Belgium.

Milisavljevic, N. (1999). Mine shape detection and data fusion

considerations. Proceedings of the Hudem symposium. RMA,

Brussels, Belgium.

Scheers, B. (1999). Development of a laboratory UWP GPR system for

mine detection. Proceedings of the Hudem symposium. RMA,

Brussels, Belgium.


An integrated robotic system for antipersonnel mines detection

Documents

Transcript of An integrated robotic system for antipersonnel mines detection