A Universal Framework for Field Robotics

Jonathan O'BrienDepartment of Civil and Environmental Engineering,

University of New South Wales,Sydney,2052

Australia

Abstract

This paper present a .general .paradigm for thedevelopment of mixed. man .and robot systems forthe .execution of industrial fieldwork operations.The theory covers both animate and .inanimatefieldwork systems and is based upon the idea ofbehaviour programmable physical.action units andmodules. The paper gives examples .of thetheory's use to model dynamic systems composedof individual systems and fleets of, disparate­form, active agents. Some emerging applicationsof mixed fleet robot systems in construction,agriculture, forestry and mining are presented.

1 IntroductionThe modem fields of agriculture, mining, forestry,shipbuilding, civil engineering, and assets maintenance,for example, are capital and machine intensive largescale industrial activities. Automation of these activitiesis difficult because any new systems must operate withinthe context of existing methods and processes and mustbe compatible with current mixed manual andmechanized operations. To date, the development ofrobot systems suited to these, non-greenfield, operationshas been hampered by a lack of a general theory of fieldrobotics in the context of practical work-doing industrialsituations. Such a theory is required to guide designersin the synthesis of suitable machinery systems to suit thereal marketplace and as well to unify efforts across whatseems to be initially quite disparate sector disciplines.

The aim of this paper is to present a, high level ofabstraction, theory of work-doing mobile roboticsystems and for the fleet use of such systems. The aimis also to show how some new ideas of field roboticscan merge seamlessly with more. traditional forms ofindustrial machinery. The paper attempts to do this bydeveloping a general theory of fieldwork systems basedupon the proposition that deliberative environmental

21

change occurs as the result of the cumulative action ofsets of behaviour programmable physical agents andpowered machines.

2 TerminologyWithin this paper the term "machine"· will be taken tomean :'a. powe~~ mechanjsm capabht of doingmechanIcal work. The notion of "a programmablemachine" is taken to mean a machine that can bereconfigured to perform different .activities .by means ofa to set of instructions. The· instructions may be giventhrough changes in hardware or in software [Mair1988]. The idea of a programmability in machines is acenturies old one and probably originated with theinvention of the Jacquard 100m in 1801 [Gunn 1982].In this paper, however, the notion of programmability isfurther ex.tended to include a machine whose behaviouris not just pre-programmed but which can be altered'on..the-go' by a series of extemal instructions. Thus, aconstruction motor. grader can be considered to be amulti-degree of freedom, blade-wielding, tractor whoseinternal geometry and power parameter can be changed,by N-control levers- as the machine is doing work (i.e.in real time).

The term 'capital' is used here in its technicaleconomic sense to mean any piece(s) of physicalapparatus that might be employed in a physicalproduction process. The word is used here to mean 'allthe tools and equipment incidental to the production ofother goods' rather than in its .everyday accountancysense of cash or money. Fieldwork then is a processrequiring people, energy and capital. Capital can befurther divided into passive tools and equipment andpowered machines.

3 Physical capital and fieldworkProcesses

Agricultural, mining, construction and forestry typeactivities nowadays involve technical processes that aregenerally capital intensive. That is, they require lar~e

amounts of specialized physical equipment and tools Inthe form of tractors, cranes, pumps, drill-systems, powersaws, electro-hydraulic jacks and so on. Thus, a typicalcivil or mining tunneling operation may involve the useof millions of dollars of rock drilling equipment,shotcreting machines, dewatering pumps,electriclocomotives and self-erecting shoring systems.Similarly, land clearing and open cut mining mayrequire extensive earthmoving activity and the use ofheavy loaders and haulage vehicles. Crop productionmay use tractors, balers, harvesters and so on. Likewise,many industrial operations require the use of heavylifting equipment and mobile cranes. Motorisedmachinery, which is controlled by people or computers,plus passive objects comprises the requisite physicalcapital an overall project or process.

4 Development of a Universal Model ofFieldwork Machines

In a previous paper [O'Brien 1997] a universal-formmodular model to cover all types of motorised fieldworkmachinery systems - both robotic and non-robotic - waspresented. Figure 1 indicates the abstract structure of anarchetypal fieldwork machine. The model includes thefunctional elements that constitute a very wide range of

Pilot signals ­given as a timeseries ofcommands

Physicalcontrols

fieldwork machines, system types and power tools.Both human workers and inanimate machinery types arecovered by the model. The abstract machine isinformation driven and real-time, behaviourprogrammable. In the model, the control system can bea human (acting in an operator-in-the-Ioop mode) or acomputer system. The model presumes a system ofexternal sensors but it can be modified, if necessary, toinclude internal sensor systems. The most common typeof operator interface .is the rate control, such as invalves and throttles. Position controls such as steeringwheels and the like, however, may also be used. Insome systems only one or two of the modules may beevident. i.e. the last module may be null. Thus, a handpower drill may only have the motor and activestructure module.

In the model, the term 'active structure' refers toany serial or parallel-topology variable geometrystructure or powered mechanism. This term coversgantries, scissors-lifts, and knuckle booms as well asbackhoes, powered augers and all forms of loadermechanism. It also applies to many other machine types-- irrespective of their particular kinematic form.. Infigure 1, the term active stricture is used in the plural.This is to allow for the fact that some fieldworkmachines, such as wHeel loaders with backhoes, cancarry two or more kinds of active device. The unit offigure 1 will typically contain a power source in theform of an energy· transducer or heat engine plus someform of complex kinematic chain.

INTERNALLYPOWEREDBEHAVIOURALSYSTEM

IContinuous­form machineaction

~

Figure 1 -- A proposed abstract-form model ofa fieldwork machine

22

5 Illustrations of the use of the ModelThe model of figure 1 is intended to cover a range ofmachines. These may range from self-propelledscissors lift and towed roller systems through tocomplex multiple degree of freedom system such asmotor graders and legged tool carriers.

An example of a fieldwork machine of complextype that can be modelled is the laser controlled,telescopic-boom, concrete screed system of figure 2.Clearly visible is the active structure plus the mobilityplatform and jack-up base fixity platform.

Figure 2. An example, multi-depee of freedom, mobile tool­handler/concrete-screed-system

6 Suggested Uses of this ModelThe main use of figure·1 is to model the action of singlerobotic machines and groups of such machines [O'Brien1997]. Thus, the schema may be used to model theinteraction of fleets of disparate-type, operatorcontrolled or robotically controlled, machines on miningsites, in agricultural operations or in forestry orconstruction sites.

AIITONOMOUS.':;:·.:coNTR,oI·:-!.··sR?'.,::;··••:,:::·;·t----------

7 An Illustrati've Use of the Model toInvestigate problems of Multi­Machine Management

A potentially very productive use of this model is theanalysis of the control processes required to maintain afleet of different machinery types operating close toeach other. In these circumstances the machines may allbe operating within a common physical area. Ofnecessity, under these, close-coupled, conditions, themachines may have to negotiate priorities and spaceusages. Also, each machine may be operatingautonomously. A common fieldwork situation wherespatio-temporal machine coordination is required, andwhere machine mutual interference occurs, is whenmaterial delivery-trucks, spreader-equipment andcompactors work together on the top of an earthworkembankment (figure 3). Another case would be wheremultiple machines have to operate within the tightconfines of a tunnel or shaft.

Figure 3- A sample multi-agent operation

Figure 4 represents a model of a three active-unitmutual interaction system of the type of figure 3

Figure 4 -- A model of a three-machine cooperative work-system

23

For coordination to occur within this organisationalstructure there must exist bi-directional communicationslinks between any inanimate actors in the system. Thediagram, however, still applies even if the machines arepeople and one is referring to a human work-team ratherthan to a machinery fleet. The diagram also coversspecial cases -- such. as when one active earthmovingmachine push-loads another.

8 MODELLING MOBILEMECHANICAL-ACTIONOPERATIONS

Spatial operator notions can be used to describe, in aconvenient and powerful manner, fieldwork processesand to set them up for simulation modelling. Forexample, consider the operation of a heavy vibratingroller (figure 6) in the compaction of earthworks. Fortechnical reasons, the action of the compaction machinein real life requires certain rolling patterns (figure 7) beadopted [Forssbladd 1981].

InItNmtw1t

Figure 6 -- Schematic of a mobile vibratory rollercompactor

P.I. 1,3,5,7, etc.

- .....~----, ........~,, "".------------ ,.>-~ ,.,'....._---------~

Figure 7 -- Rolling pattern for earthworks compaction

Whilst an earthworks example has been used here toillustrate the concept, the same basic model could beused equally for a wheat harvester process or a snow­clearing process.

If we view the compactor machine as a both spaceand intensity programmable mechanical action machine,we can then represent this resource by a "token" appliedto a "place". If we do this we can then developoperations-maps such as that illustrated in figure 8. Wecan also develop discrete-event, state-model simulationsof the spatio/temporal behaviour of such systems. Onemethod that shows much promise in this regard is themethod of Petri-Nets [Wakefield et al 1997].Alternatively, we can look at how a machine must beprogrammed in space-time to generate a particularbehaviour or to interact with ·another spatio-temporallyprogrammed machine or smart-agent.

24

In figure 8, behaviour through time can be furthermodelled by assigning. dwell times to tokens in cells orby developing the cells as voxels. That is, bydeveloping a time axis at right angles to the plane of

.figure 8. Thus, one can have a 'cube' of 'places' as atype of finite state machine through which tokens can'migrate' as a function of time. The intensity of actionof the operator at each place however has to bemodelled as some variable property of the operator.

.-+. -.. -... -~ 1-~.,~0 .--+

0 ~,/

•e'"- ....-....-e +0- +- •

Figure 8 -- A token-place model of the behaviour of avariable-action mechanical operator

From these ideas and from those of on-the-runprogrammability, it is clear that one can programmachines to execute "behaviours"- as high orderplanning units - rather than viewing machines only interms of low-order activities like programming in 'joint­space'.

9 Robots as programmable machinesFrom the above discussions, if we define a 'robot' as abehaviour programmable fieldwork machine and if wesee most traditional· fieldwork machines as operator-in­the-loop programmable systems then it would now seemclear that there is no essential difference betweenfieldwork robots and existing fieldwork machinerysystems. The only real difference is in the mode ofprogramming. The author has also argued previouslythe essential similarity between cranes, manipulatorsand standard manufacturing robots [O'Brien 1995,O'Brien 1991]. This idea has now been extended toagricultural machines, forestry machines and miningequipment. At an abstract level, they are all the same

10 Development of a universalcontroller box.

One impo:rtafit implication of this programmablefieldwork machine theory is that, under-the-skin, allfieldwork machines are the same -- in that they may becontrolled by information signals. This suggests that it .is possible to build one only, universal fieldworkmachine controller box that can fit all machines and allsituations. This single box idea has great potentialeconomic value - in that it means that earthmovingmachine developers, autonomous truck developers,

crane machinery developers and agricultural machinerydevelopers can share the same, high level, intelligentfieldwork machine controller box. This observationsuggests that major economies of scale can bedeveloped and that for the same amount of money verymuch more sophisticated general purpose controllerscan be developed than is possible for one-off projectdevelopments. This notion has been previouslydiscussed in some detail in [013rien 1997]

11 Modeling mixed man-machineoperations and activities.

In relation to figure 1, it was suggested that humanaction and robotic machinery are essentially of the samenature. From this observation, it becomes evident thatusing robotic systems programming concepts (such asdirect joint programming or through the use. of teachpoints and inverse kinematics) it is possible to modeland simulate human action in a manner and to a degreeof accuracy hitherto impossible.

Figure 9a - Human activities dynamically modelled usingprogrammable machine concepts

Figure 9a, for example, shows a worker, modeled as ahumanoid robot, in the process of setting up a ladder towash a window. Figure 9b shows the worker wiping thewindow with a rubber blade. Figure 6.2 shows the typeof programmable skeleton idea that was used.

The dynamic simulation of the constructionprocedure shown was developed by the author and MikeGoodwin of Deneb Robotics, Australia.

25

Figure 9b -A sequence from a window washing simulation

12 COMPUTER INTEGRATED ROADCONSTRUCTION

Following figure 4, an example of the use ofmultiple machines communicating through a shared database is that of a fleet of"asphalt paving machines in theEuropean CIRC (Computer Integrated RoadConstruction project) [Peyret 1998]. In this case realtime positioning equipment is used to develop ageographical information system that may be shared viaradio link across a fleet of machines.

The equipment is now available commercially andFigure 10 shows a full scale road compactor equippedwith driver assisted real-time radio linked display

Figure 10 - On machine instrumentation system anddisplay

Moving forward

Figure 11 - Real time GIS and QAlQC system foran asphalt paving operation

Figure 11 shows the cumulative action of thecompaction machine on a section of roadway, withcomparison to a set of specified design procedures. Thevehicle shown is controlled by a composite GPS andlocal laser [3D] kinematic positioning system to anaccuracy of 1 cm in plan and 2 em in elevation. Thesystem can operate up to a speed or around 10 kIn perhour.

15 ConclusionsIn this paper, a universal view of mixed man-machine­robot production and work systems has been presented.This Vlew allows the design of integrated production

26

systems wherein eXIStIng manual processes, operatorcontrolled machinery procedures and computercontrolled machinery procedures can be embraced by asingle general schema. Such an integrating schema isconsidered necessary to permit the gradual, andappropriate, introduction of robot and automationconcepts into traditional agricultural mining,construction and forestry operations. Through thisintegrating schema it can be seen that manualoperations, mechanisation and robotic automation aredevelopments that lie along a single continuum and thatthe developments merge seamlessly one into the other.

References[Mair 1988] Gordon Mair Industrial Robotics Prentice

Hall 1988.[Gunn 1982] Thomas Gunn. The Mechanization of

Design and Manufacture. Scientific American Sept1982, pages 87-108.

[O'Brien 1977] J. B. O'Brien Behaviour programmablemachines for fi con ction operations,Intemational ce on Field and ServiceRobotics The Australian. NationalUniversi erra, Australia, 8-10 December1997, pa to 33

[Forssbladd 1 . Forssbladd. Vibratory Soil andRockfil io napak Maskin AB, 1981

[Wakefield 9 Ronald Wakefield, JirawatDamrian Jonathan O'Brien. A Petri NetBased S for the Modeling and ComputerSimu Au onstruction Operations.In Proce SARC, Pittsburgh, USA,

. June, 19 on University.[O'Brien 1995 ' rien. Large Scale, Computer-

Controlled, an lato For.Field BasedApplications 26th International Symposium onIndustrial Robotics (ISIR) Singapore, October 4-61995 pages 79-84. 1996

[O'Brien 1991] J. B. O'Brien Development of macro­manipulator based robotic systems for generalconstruction use Proc. 8th ISARC - Stuttgart 3-5June 1991 page 431-438.

[Winter 1996] R.L Winter and J. B. O'Brien SelfProgramming Cqnstruction Machines Proc. 13thISARC, Tokyo, June 11-13, 1996, Japan RobotAssoc. pages 635-644.

[Deneb 1998] http://www.deneb.com[Winter 1999] R.L. Winter. A new Approach to the

Design of Autonomous Construction Agents.University of NSW, PhD Thesis 1999 (in press).

[Peyret 1998] Francois Peyret Computer Integrated··road Construction 15th ISARC Munich 1998