Articles

16 AI MAGAZINE

“With autonomy we declare that no sphere is off limits.We will send our spacecraft to search beyond the horizon, accepting that we cannot directly control

them, and relying on them to tell the tale.”

– Bob Rasmussen New Millennium Autonomy Team

AI Magazine Volume 17 Number 3 (1996) (© AAAI)

■ A new generation of sensor-rich, massively dis-tributed, autonomous systems are being devel-oped that have the potential for profound social,environmental, and economic change. These sys-tems include networked building energy systems,autonomous space probes, chemical plant con-trol systems, satellite constellations for remoteecosystem monitoring, power grids, biospherelikelife-support systems, and reconfigurable trafficsystems, to highlight but a few. To achieve highperformance, these immobile robots (or im-mobots) will need to develop sophisticated regu-latory and immune systems that accurately androbustly control their complex internal func-tions. Thus, immobots will exploit a vast nervoussystem of sensors to model themselves and theirenvironment on a grand scale. They will usethese models to dramatically reconfigure them-selves to survive decades of autonomous opera-tion. Achieving these large-scale modeling andconfiguration tasks will require a tight couplingbetween the higher-level coordination functionprovided by symbolic reasoning and the lower-level autonomic processes of adaptive estimationand control. To be economically viable, they willneed to be programmable purely through high-level compositional models. Self-modeling andself-configuration, autonomic functions coordi-nated through symbolic reasoning, and composi-tional, model-based programming are the threekey elements of a model-based autonomous sys-tem architecture that is taking us into the newmillennium.

The limelight has shifted dramatically inthe last few years from an AI traditionof developing mobile robots to that of

developing software agents (that is, softbots)(Etzioni and Weld 1994; Kautz et al. 1994;Maes and Kozierok 1993; Dent et al. 1992; Et-zioni and Segal 1992). This shift is reflected inHollywood, which has replaced R2D2 with cy-berpunk descendants of Disney’s TRON as thepopular robot icon. One motivation for theshift is the difficulty and cost of developingand maintaining physical robots, which arebelieved to substantially impede progress to-

ward AI’s central goal of developing agent ar-chitectures and a theory of machine intelli-gence (Etzioni and Segal 1992). As Etzioni andSegal argue, software environments, such as aUNIX shell and the World Wide Web, providesoftbots with a set of ready-made sensors (forexample, LS and GOPHER) and end effectors (forexample, FTP and TELNET) that are easy tomaintain but still provide a test bed for ex-ploring issues of mobility and real-time con-straints. At the same time, the recent Internetgold rush and the ensuing web literacy hasprovided an enormous textual corpus thatscreams for intelligent information-gatheringaides (Levy, Rajaraman, and Ordille 1996;Knoblock and Levy 1995).

However, two concerns have been raisedabout using software agents as a research testbed and application domain: First, softbotsoften operate in an environment lacking therich constraints that stem from noisy, analogsensors and complex nonlinear effectors thatare so fundamental to physical environments.Can such a software environment adequatelydrive research on agent kernels? Second, giv-en that much of the information on the In-ternet is textual, will oft-envisioned softbotapplications, such as information gatheringand synthesis, be viable before the hard nutof language understanding has been cracked?

In this article, we argue that the informa-tion-gathering capabilities of the Internet, cor-porate intranets, and smaller networked com-putational systems supply additional test bedsfor autonomous agents of a different sort.These test beds, which we call immobilerobots (or immobots), have the richness thatcomes from interacting with physical environ-ments yet promise the ready availability asso-ciated with the networked software environ-ment of softbots. Potential immobile robotsinclude networked building energy systems,autonomous space probes, chemical plantcontrol systems, satellite constellations forecosystem monitoring, power grids, biosphere

Articles

FALL 1996 17Copyright © 1996, American Association for Artificial Intelligence. All rights reserved. 0738-4602-1996 / $2.00

Immobile RobotsAI in the New Millennium1

Brian C. Williams and P. Pandurang Nayak

We argue thatthe focus ofattention of

immobilerobots is di-

rected inward,toward

maintainingtheir internalstructure, in

contrast to thefocus of

traditionalrobots, whichis toward ex-ploring and

manipulatingtheir externalenvironment.

nel for controlling an immobot’s regulatorysystem. Our work on these systems fuses re-search from such diverse areas of AI as model-based reasoning, qualitative reasoning, plan-ning and scheduling, execution, propositionalsatisfiability, concurrent reactive languages,Markov processes, model-based learning, andadaptive systems. MORIARTY and LIVINGSTONE aregrounded in two immobot test beds. MORIARTY

was part of the RESPONSIVE ENVIRONMENT (Elrodet al. 1993; Zhang, Williams, and Elrod 1993),an intelligent building control system devel-oped within the Ubiquitous Computing Pro-ject at Xerox PARC. LIVINGSTONE is part of theREMOTE AGENT, a goal-directed, fully au-tonomous control architecture, which will flythe National Aeronautics and Space Adminis-tration (NASA) Deep Space One space probe in1998 (Pell, Bernard, et al. 1996). At least oneof these immobots has the promise ofsignificant impact at the beginning of the newmillennium.

Examples of Immobile RobotsHardware advances in cheap single-chip con-trol and communication processors, sensors,point actuators, analog-to-digital converters,and networking have enabled a new categoryof autonomous system that is sensor rich,massively distributed, and largely immobile.This technology is being quickly embedded inalmost every form of real-time system, fromnetworked building energy systems to space-craft constellations. The current generation ofthese hybrid hardware-software systems hascreated a slumbering giant whose potentialhas only begun to be tapped. Furthermore,they offer a diverse set of ready-made immo-bile-robot test beds just waiting to be exploit-ed by AI researchers.

The potential of this technology capturedthe imagination of Hollywood in the 1960swith such movies as 2001: A Space Odyssey,The Forbin Project, and The Andromeda Strainand the Star Trek television episode “Spock’sBrain.” The most famous computational in-telligence, the HAL9000 computer, and themost famous biological intelligence, Spock,were both for a time immobile robots. In thisarticle, we use HAL and Spock as startingpoints for two immobile robot futures.

HAL9000

In the movie 2001, HAL is the first of a newgeneration of computers with unprecedent-ed intelligence and flawless behavior. HAL ischaracterized as the single mission elementwith the greatest responsibility, the “brain

life-support systems, and reconfigurable trafficsystems. Conversion of these and other real-time systems to immobile robots will be adriving force for profound social, environ-mental, and economic change. The power of aterm such as robot, mobot, or softbot is theway in which it sets our imagination free. Thepurpose of this article is to see where the im-mobot metaphor can take us, from sciencefiction to fundamental AI insights tosignificant applications.

We argue that the focus of attention of im-mobile robots is directed inward, towardmaintaining their internal structure, in con-trast to the focus of traditional robots, whichis toward exploring and manipulating theirexternal environment. This inward directionfocuses the immobot on the control of itscomplex internal functions, such as sensormonitoring and goal tracking; parameter esti-mation and learning; failure detection andisolation; fault diagnosis and avoidance; andrecovery, or moving to a safe state. Metaphor-ically speaking, the main functions of an im-mobot correspond to the human nervous,regulatory, and immune systems rather thanthe navigation and perceptual systems beingmimicked in mobile robots.

Finally, we argue that these immobots giverise to a new family of autonomous agent ar-chitectures, called model-based autonomous sys-tems. Three properties of such systems are cen-tral: First, to achieve high performance,immobots will need to exploit a vast nervoussystem of sensors to model themselves andtheir environment on a grand scale. They willuse these models to dramatically reconfigurethemselves to survive decades of autonomousoperations. Hence, self-modeling and self-configuration make up an essential executivefunction of an immobot architecture. Second,to achieve these large-scale modeling andconfiguration functions, an immobot architec-ture will require a tight coupling between thehigher-level coordination function providedby symbolic reasoning and the lower-level au-tonomic processes of adaptive estimation andcontrol. Third, to be economically viable, im-mobots will have to be programmable purelyfrom high-level compositional models, sup-porting a “plug and play” approach to soft-ware and hardware development.

These properties of a model-based au-tonomous system are embodied in two imple-mented systems: MORIARTY (Williams and Mil-lar 1996) and LIVINGSTONE (Williams andNayak 1996). LIVINGSTONE provides a kernel forcontrolling the immune system of immobilerobots, and MORIARTY provides part of the ker-

Articles

18 AI MAGAZINE

… in 1995,NASA putforth thechallenge ofestablishing a virtual presence inspace throughan armada ofintelligentspace probesthat au-tonomouslyexplore thenooks andcrannies ofthe solar system .…

and central nervous system” of a spacecrafttargeted for Jupiter.

It might seem puzzling to call HAL an im-mobot; after all, it travels from Earth toJupiter in 18 months, making it the fastestmanmade artifact of its time. However, tradi-tional mobile robots typically focus on theirexternal environment and on the tasks ofnavigation and obstacle avoidance. In con-trast, the movie portrays HAL with an extremesense of immobility. The camera zooms in forlong periods on HAL’s optical sensors, whichare attached to walls throughout the space-ship. HAL’s actuators are also extremely limit-ed; examples include the opening and closingof doors and the raising and lowering of beds.The distribution of these sensors and actua-tors throughout the spacecraft compensatesfor their immobility, giving HAL a sense ofomnipresence. HAL’s attention is focused in-ward throughout the movie, highlighted bythe characterization of HAL as the “brain andcentral nervous system” of the spacecraft asopposed to the navigator and pilot. HAL’s ma-jor responsibility is to look after the health ofthe spacecraft and crew, including monitor-ing the spacecraft’s health, performing faultdiagnosis and repair, operating the life-sup-port systems, and continuously monitoringthe medical status of crew members in hiber-nation. Finally, the movie highlights the con-nection between HAL’s higher-level symbolicreasoning and the complex, low-level, auto-nomic processes distributed throughout thespacecraft. Hence, HAL can be thought of asthe spacecraft’s immune system.

The New Millennium Program Although many aspects of 2001 now seemfarfetched, the current future of space explo-ration is no less exciting. With the creation ofthe New Millennium Program in 1995, NASAput forth the challenge of establishing a vir-tual presence in space through an armada ofintelligent space probes that autonomouslyexplore the nooks and crannies of the solarsystem: “With autonomy we declare that nosphere is off limits. We will send our space-craft to search beyond the horizon, acceptingthat we cannot directly control them, and re-lying on them to tell the tale” (Rasmussen1996).

This presence is to be established at anApollo-era pace, with software for the firstprobe to be delivered in mid-1997, leavingonly 1-1/2 years for development, and theprobe (Deep Space One) to be launched inmid-1998. The additional constraint of lowcost is of equal magnitude. Unlike the billion-

dollar Galileo and Cassini missions with hun-dreds of members in their flight operationsteams, New Millennium Program missionsare to cost under $100 million, with onlytens of flight operators. The eventual goal is a$50 million spacecraft operated by a merehandful of people. Achieving these goals willrequire autonomy and robustness on an un-precedented scale.

The final challenge, spacecraft complexity,is equally daunting. Consider Cassini, NASA’sstate-of-the-art spacecraft headed for Saturn.Cassini’s “nervous system” includes a sophis-ticated networked, multiprocessor system,consisting of two flight computers that com-municate over a bus to more than two dozencontrol units and drivers. This nervous sys-tem establishes complex sensing and controlpaths to an array of fixed sensors and actua-tors, including inertial reference units, sunsensors, pressure sensors, thrusters, main en-gines, reaction wheels, and heaters. The mostcomplex is the main engine subsystem, con-sisting of a web of redundant pipes, valves,tanks, and engines that offer exceptional lev-els of reconfigurability. Coordinating thesecomplicated, hybrid systems poses significanttechnical hurdles.

Together, the goals of the New MillenniumProgram pose an extraordinary opportunityand challenge for AI. As with HAL, our chal-lenge is to provide the immune system forthis type of immobile robot over the lifetimeof its mission.

“Spock’s Brain” A second example of an immobile robot from1960’s Hollywood comes from a little-knownepisode of the original Star Trek series called“Spock’s Brain.” In this episode, Spock’s bodyis found robbed of its brain by an unknownalien race, and the crew of the Enterprise em-barks on a search of the galaxy to reuniteSpock’s brain and body. Spock detects that hehas a new body that stretches into infinityand that appears to be breathing, pumpingblood, and maintaining physiological tem-perature. When discovered, Spock’s brain isfound to be within a black box, tied in bylight rays to a complex control panel. Insteadof breathing, maintaining temperature, andpumping blood, he is recirculating air, run-ning heating plants, and recirculating water.That is, the function that requires thissupreme intelligence is the regulation of aplanetwide heating and ventilation system.

As with HAL, Spock’s body in this case is ex-tremely immobile. The episode portrays animmobile robot as a massively distributed be-

Articles

FALL 1996 19

management systems. These are networkedmultiprocessor systems containing hundredsto thousands of processors that allow high-fidelity sensing and control throughout abuilding. They will enable systems that are farmore energy efficient and at the same timeimprove indoor air quality. Second is the in-terconnection of building and home utilitiesthrough optical fiber networks, which has ledto several commercial alliances betweenmembers of the utility and telecommunica-tions industries. Third is the deregulation ofthe utilities and the ensuing establishment ofcomputer-based energy markets. These newenergy markets will allow peak and averageelectric rates to be fluidly adjusted, enablingmore even and stable balancing of the energyload. Technologies for storing energy withinbuildings during off-peak hours will enableeven greater stability and efficiency. The rapiddeployment of these networked control tech-nologies has already generated a multibillion

hemoth, sufficient to encircle a globe. Again,the crucial link between high-level reasoning(that is, Spock’s brain) and autonomic pro-cesses is highlighted. Finally, although 2001highlights HAL’s function as an immune sys-tem that maintains the health of the im-mobot, this episode highlights Spock’s func-tion as a regulatory system, performinghigh-fidelity control of the immobot’s inter-nal organs. A regulatory system provides anefficient metabolism, supplying central re-sources efficiently, equitably distributingthese resources throughout the system, con-suming these resources efficiently, and atten-uating the effects of disturbances.

The Responsive Environment The concept in “Spock’s Brain” is not as im-plausible as it might seem. In fact, it is quick-ly becoming a reality through a confluence offorces (figure 1). First is the broad installationof the new generation of networked building

Articles

20 AI MAGAZINE

Figure 1. Immobots of Unprecedented Size Are Being Developed in the Area of Building Energy Management.Key elements include networked building control systems, optical networks that connect buildings on a power grid to multiple power-

production facilities, deregulation of the power industry, and the establishment of a dynamic energy market.

Structurally,immobots are massivelydistributed,physicallyembedded,autonomoussystems witha large arrayof simple,fixed-locationsensors andactuators.Functionally,an immobot’sprimary taskis to controlits massiveregulatory,immune, and nervoussystemsthrough acoupling ofhigh-level reasoning andadaptive autonomicprocesses.

dollar growth in the service industries for re-mote building monitoring and maintenance.However, the potential of these immobilerobots for optimal energy-efficient controlhas yet to be tapped.

These are only two examples of an ever-in-creasing collection of immobile robot testbeds being constructed. NASA’s Earth-observ-ing system is moving toward a level of sens-ing that will enable full Earth ecosystemmodeling, providing insight into problems ofpollution, global warming, and ozone deple-tion. Vast networked control systems are gen-erating a revolution in chemical processing,factory automation, drug manufacturing, andsemiconductor fabrication, producing immo-bile robots that enable substantial improve-ments in quality, efficiency, and safety.

These are not far-off dreams. Many govern-ment and private institutions, such as NASA,Pacific Gas and Electric, Rockwell Automa-tion, Echelon, Hewlett Packard, and JohnsonControls, are aggressively embracing thesetechnologies as fundamental to their futurein the new millennium, that is, in the imme-diate future. The two immobot test beds dis-cussed in this article represent first steps to-ward these future visions.

Immobot Characteristics From these examples, we can extract a num-ber of properties that distinguish immobotsfrom their mobile robot and softbot siblings,both in terms of their physical structure andtheir most salient functions. Structurally, im-mobots are massively distributed, physicallyembedded, autonomous systems with a largearray of simple, fixed-location sensors and ac-tuators. Functionally, an immobot’s primarytask is to control its massive regulatory, im-mune, and nervous systems through a cou-pling of high-level reasoning and adaptiveautonomic processes. More specifically, animmobot has the following distinctive fea-tures:

Physically embedded: An immobot’s sen-sors and actuators operate on the physicalworld, many of which operate in noisy envi-ronments. Immobots need to react in realtime; however, the time scale of the reactiontime can vary dramatically. For example, aspacecraft might have to act instantaneouslyto close off a leaking thruster but then mighthave weeks during its cruise to perform a self-diagnosis.

Immobile: An immobot’s sensors and actu-ators reside at fixed locations and are largelylimited to one-dimensional signals and one-

degree-of-freedom movement. Examples in-clude light sensors, window-blind actuators,gyroscopes, valves, and reaction wheels. Al-though an immobot’s hardware is often re-dundant and highly reconfigurable, the prim-itive elements of interaction that are used tobuild a configuration are simple and fixed. Incontrast, a traditional robot typically has a setof complex mobile sensors and actuators,such as three-dimensional vision systems andarticulated hands, arms, and legs.

Omnipresent: The lack of sensor and actu-ator mobility is compensated for by the sheernumber of sensors and actuators. A spacecrafthas dozens; a building can have on the orderof thousands.

Sensors and actuators must reside a prioriin all locations that the immobot wants tomodel or affect, giving the immobot the po-tential for being continuously aware of all as-pects of its environment. In contrast, a mo-bile robot’s few precious sensors andactuators must be navigated to the pointwhere they are needed. The omnipresence ofthe immobot shifts the bottleneck from theexpense of gathering each piece of informa-tion to the integration of a continuousstream of data.

Massively distributed and tightly cou-pled: Computation is distributed alonglengthy communication paths, with increas-ingly complex computation being performedat sites near sensors and actuators. For exam-ple, in Cassini, a complex set of physical andsoftware processes work together to fire anengine: The flight computer turns on an en-gine by sending a command over a bus to acommunication processor, which tells a driv-er to open two valves, which causes fuel toflow into the engine, where combustion oc-curs. These properties lead to distributed sens-ing and control problems of high dimension-ality as a result of tight couplings betweeninternal components. This high di-mensionality makes these problems extreme-ly difficult to solve.

Self-absorbed: Mobile robots and softbotsfocus largely on what is occurring in theworld outside the “bot,” including navigat-ing around obstacles, moving through net-works to databases, and changing navigationpaths because of external failures. However,an immobile robot’s attention is largely di-rected inward toward monitoring the inter-nal health of its network (immunology) andreconfiguring its components or controlpolicies to achieve robust performance (regu-lation). Although some reasoning is directedoutward, the external world is not fluid in

Articles

FALL 1996 21

responses to all possible situations quickly be-comes intractable. Tractability is typically re-stored with the use of simplifying assumptionssuch as using local suboptimal control laws,assuming single faults, ignoring sensor infor-mation, and ignoring subsystem interactions.Unfortunately, this results in systems that areeither brittle or grossly inefficient.

Finally, the autonomic processes of an im-mobot involve a broad range of discrete, con-tinuous, and software behaviors, includingcontrol laws, Kalman filters, digital hardwarecommands, and software drivers. This rangeof behaviors needed by an immobot makes itdifficult and expensive to both manually syn-thesize high-fidelity autonomic processes andcouple these autonomic processes to high-level symbolic reasoning.

Model-Based Autonomous Systems

The goal of our research program is to solvethese difficulties by developing a model-basedautonomous system kernel for maintainingthe regulatory and immune systems of im-mobots. The kernel that we are driving to-ward is defined by three desiderata: (1) mod-el-based programming, (2) model-basedreactive execution, and (3) model-based hy-brid systems.

Our work on immobots originated withwork on interaction-based design (Williams1990), which explored the coordination andconstruction of continuous interactions, as adesign task that involves the addition of nov-el hardware-device topologies. In contrast,model-based autonomy explores the coordina-tion of hardware and software interactionsusing a digital controller.

Model-Based ProgrammingA model-based autonomous system addressesthe difficulty of reasoning about systemwideinteractions using model-based programming.Model-based programming is based on theidea that the most effective way to amortizesoftware development cost is to make the soft-ware plug and play. To support plug and play,immobots are programmed by specifyingcomponent models of hardware and softwarebehaviors. A model-based autonomous systemcombines component models to automate allreasoning about systemwide interactions nec-essary to synthesize real-time behaviors suchas the ones in figure 2. The development ofmodel libraries is used to reduce design time,facilitate reuse, and amortize modeling costs.

The application of a stringent qualitative

the same sense that it is for classical robots.One of a kind: Ironically, what binds to-

gether immobile robots is that no two arealike. Drug manufacturing lines, Disney’sspace mountain, car factories, Mars rovers,Earth-orbiting satellites, deep-space probes,and Antarctic biospheres are each one-of-a-kind devices, making it difficult to amortizetheir development costs. The dilemma ishow to cost effectively build these one of akinds yet provide high performance and reli-ability.

Controlling Immobots In the preceding sections, we argued that thedominant function of an immobot is to con-trol its massive regulatory, immune, and ner-vous systems through a coupling of high-lev-el reasoning with adaptive autonomicprocesses. Providing a regulatory and im-mune system involves a broad set of tasks, in-cluding those listed in figure 2.

Writing software to control the immuneand regulatory systems of an immobot isdifficult for a variety of reasons: First, it oftenrequires the programmer to reason throughsystemwide interactions to perform the appro-priate function. For example, diagnosing afailed thruster requires reasoning about the in-teractions between the thrusters, the attitudecontroller, the star tracker, the bus controller,and the thruster valve electronics. The com-plexity of these interactions can lead to cogni-tive overload, causing suboptimal decisionsand even outright errors. Furthermore, theone-of-a-kind nature of immobots means thatthe cost of reasoning through systemwide in-teractions cannot be amortized and must bepaid over again for each new immobot.

Second, the need for fast reactions inanomalous situations has led to a tradition ofprecomputing all responses. However, im-mobots often operate in harsh environmentsover decades, so that explicitly enumerating

Articles

22 AI MAGAZINE

Parameter estimation Adaptive controlMonitoring Control policy coordinationMode confirmation Hardware reconfigurationGoal tracking Fault recoveryDetection of anomalies StandbyIsolation of faults SafingDiagnosis of causes Fault avoidanceCalibration

Figure 2. Tasks Performed to Maintain the Immobots Regulatory and Im-mune Systems.

The goal ofour researchprogram is… develop-ing a model-based autonomoussystem kernel formaintainingthe regulatoryand immunesystems ofimmobots.

modeling methodology is used to reduceboth modeling time and the sensitivity tomodel inaccuracies and hardware changes.Our experience and those of others (Malikand Struss 1996; Hamscher 1991) has shownthat extremely weak qualitative representa-tions, for example, representing only devia-tions from nominal behavior, are sufficientfor many model-based autonomy tasks.Counter to the folklore of the field (Sacks andDoyle 1992), ambiguity and intractablebranching has not proven to be a significantpractical issue.

Model-based programming on a large scaleis supported by developing languages, com-pilers, debuggers, and visualization tools thatincorporate classical concepts of object-ori-ented, procedural, and hierarchical abstrac-tions into the modeling language.

Model-Based Reactive ExecutionThe difficulty of precomputing all responsesmeans that a model-based autonomous sys-tem must use its models to synthesize timelyresponses to anomalous and unexpected situ-ations at execution time. Furthermore, theneed to respond correctly in time-critical andnovel situations means that it must performdeliberative reasoning about the model with-in the reactive control loop. Although the listof tasks that the model-based execution sys-tem must support is seemingly diverse (figure2), they divide into two basic functions: (1)self-modeling and (2) self-configuring.

Self-modeling: Identifying anomalous andunexpected situations and synthesizing cor-rect responses in a timely manner require animmobot to be self-modeling. Although partsof its model are provided a priori using mod-el-based programming, other parts need to beadapted or elaborated using sensor informa-tion. Self-modeling tasks include trackingmodel parameters over time (for example,base-line voltages), tracking changes in com-ponent behavioral modes (for example, avalve going from open to closed and then tostuck closed), and elaborating quantitativedetails of qualitative models (for example,learning a quantitative pipe model given thatflow is proportional to the pressure drop).Self-modeling tasks are listed in the left-handcolumn of figure 2; all but estimation, moni-toring, and calibration involve identifyingdiscrete behavioral modes of the immobot’scomponents.

Self-configuring: To provide immune andregulatory systems, an immobot must be self-configuring; it must dynamically engage anddisengage component operating modes and

adaptive control policies in response tochanges in goals, the immobot’s internalstructure, and the external environment. Theright-hand column of figure 2 lists tasks thatinvolve self-configuration. The first two itemsinvolve coordinating or adjusting continuouscontrol policies, and the remainder involvechanges in component behavioral modes.

Model-Based Hybrid SystemsGiven the wide range of digital, analog, andsoftware behaviors exhibited by an immobot,developing a model-based approach for coor-dinating the immobot’s autonomic processesrequires a rich modeling language and rea-soning methods that go well beyond thosetraditionally used in qualitative and model-based diagnosis. More specifically, a model-based autonomous system must be able torepresent and reason about the following:

Concurrent software: Coordinating, in-voking, and monitoring real-time software re-quires formal specifications of their behavior.These behaviors are modeled by incorporat-ing formal specifications of concurrent transi-tion systems into the model-based program-ming language. Concurrent transitionsystems provide an adequate formal seman-tics for most concurrent real-time languages(Manna and Pnueli 1992).

Continuous adaptive processes: Achiev-ing high fidelity requires the merging of sym-bolic model-based methods with novel adap-tive estimation and control techniques (forexample, neural nets). Specifications of adap-tive processes must be combined withspecifications of discrete concurrent behaviorwithin the models. For high-level reasoningsystems that coordinate adaptive processes,the most suitable specification of the adaptiveprocesses are often qualitative.

Stochastic processes: Inherent to support-ing an immobot’s regulatory and immunefunctions is the modeling and control ofstochastic events that occur within an im-mobot’s components. This stochastic behav-ior is modeled by extending the concurrenttransition-system model, mentioned previ-ously, to modeling concurrent, partially ob-servable Markov processes.

The previous three subsections outlinedthe key requirements for model-based auton-omy. The essential property that makes theserequirements manageable is the relative im-mobility of our robots. The system interac-tions are relatively fixed and known a priorithrough the component models and their in-terconnection. The flexibility within the sys-tem is largely limited to changes in compo-

Articles

FALL 1996 23

ter, where the flow rate of the water is con-trolled by the reheat-valve position.

Regulating a building is difficult becausethe number of control variables is over-whelming, and the control variables are high-ly coupled. For example, the energy con-sumption of the fan and the chiller arenonlinear functions of the fan speed and thechange from outside to inside temperature.The optimal settings of the damper and re-heat valve then depend on the outside tem-perature and the demands that other officesare placing on the chiller and the fan. Hence,the performance of all the offices is coupled,which is exacerbated by the slow responsetime of office temperature change, making atrial-and-error approach to global control ex-tremely unstable.

We solve this problem with a gradient-de-scent controller that uses a global model toadaptively predict where the optimum lies.An informal evaluation using the responsiveenvironment test bed suggests that energysavings in excess of 30 percent are conceiv-able with help from model-based control(Zhang, Williams, and Elrod 1993).

Generic thermal models are available forcomplete buildings (for example, the DOE2simulator models) that suffice for this type ofcontrol. These models are extremely rich, in-cluding not only the building’s hardware andnonlinear thermal characteristics but alsosunlight and shading, external weather, andthe occupants’ desires. They have on the or-der of thousands of equations for buildingswith 100 or more offices.

The labor-intensive task is tailoring thismodel to the specifics of a building, which re-quires estimating numeric values for parame-ters such as thermal conductance through thewalls, heat output of the equipment, andthermal capacity of the office airspace. An im-mobot can estimate its parameters by adjust-ing their values until the model best fits thesensor data. More precisely, given sensed vari-ables y and x (a vector) and a vector of pa-rameters p, we first construct an estimator ffrom the model that predicts y given x and p:

y = f(x; p) .

Next, given a set of (x,y) data D, estimating pusing least squares fit of f to y involves solv-ing the optimization problem

p* = arg ^<yi, xi> ∈ D

(yi – f(xi; p))2.

Such an estimate is performed by an adaptivenumeric algorithm that can be viewed as oneof the immobot’s autonomic processes.

There are far too many parameters in a

minp

nent modes and control policies and adjust-ments to parameter values. In the rest of thisarticle, we consider two implemented systemsthat exploit immobility to achieve thesedesiderata. They form two major componentsof a kernel we envision for maintaining theregulatory and immune systems of im-mobots.

Responsive Environments The scenario in “Spock’s Brain” of a heatingand cooling system on a planetary scale high-lighted three aspects of immobile robots thatwe examine technically in this section: Firstis the task of maintaining a massive, high-performance regulatory system. Second is theneed by this system to acquire and adapt ac-curate models of itself and its environment toachieve high performance. Third is the needfor high-level reasoning to coordinate a vastset of autonomic processes during adaptivemodeling and regulation.

As a stepping stone toward this ambitiousscenario, we developed a test bed for fine-grained sensing and control of a suite ofoffices at Xerox PARC called the responsive en-vironment (Elrod et al. 1993). This test bed in-cludes a networked control system for thecomplete building plus 15 model offices, eachof which has been enhanced with a net-worked microprocessor that controls an arrayof sensors and actuators. These include sen-sors for occupancy, light level, temperature,pressure, and airflow and actuators for regu-lating air temperature and airflow.

The heart of the building is the centralplant, which generates airflow and cold andhot water through a fan, chiller, and boiler,respectively. The extremities of the buildingare the hundred or more offices, whose tem-perature must be regulated carefully. Theveins and arteries of the building are thepipes and duct work that deliver air and wa-ter to the extremities, where they are used toregulate temperature. The connection be-tween the central plant and a single office,used for cooling, is shown in figure 3. Officetemperature is controlled through heat flowinto, or out of, the office, which includesheat flowing from the sun and equipment,through the walls and doorways, andthrough the air duct. Heat flow into an officeis controlled by the air duct in two ways:First, the amount of airflow is controlled by adamper that partially blocks airflow. Second,the temperature of the airflow is changed byblowing the air over a radiatorlike device,called a reheat, that contains hot or cold wa-

Articles

24 AI MAGAZINE

building model to estimate all of them atonce. Instead, the immobile robot must auto-mate a control engineer or modeler’s expertiseat breaking a model into a set of tractable pa-rameter-estimation tasks and then coordinat-ing and combining their results.2 This coordi-nation represents the link between high-levelreasoning and autonomic processes in our self-modeling immobot. An army of modelers isused for large-scale tasks, such as earth ecosys-tem modeling, at great cost. This army mustbe automated for many immobile robot tasksif high performance and robustness are to beachieved. MORIARTY automates key aspects ofhow a community of modelers decompose,simplify, plan, and coordinate large-scale mod-el-estimation tasks through a technique calleddecompositional, model-based learning (DML).

Coordinating Adaptive Model Estimation

When decomposing a model into a set of esti-mation tasks, there is often a large set of pos-sible estimators (y = f(x; p)) to choose from,and the number of parameters contained ineach estimator varies widely. MORIARTY decom-poses a model into a set of simplest estima-tors that minimize the dimensionality of thesearch space and the number of local mini-ma, hence improving learning rate and accu-racy. Each estimator, together with the appro-priate subset of sensor data, forms a primitiveestimation action. MORIARTY then plans the or-dering and the coordination of informationflow between them.

To estimate the parameters for a singleoffice, MORIARTY starts with a thermal model

Articles

FALL 1996 25

optimalcontrol

s u

p

Chiller

FSPLYFan T EXT

TRM

XDMPR

TSPLY

XRHT

(model parameters)

thermalmodel

Figure 3. Model-Based Air Conditioning of a Single Office, Using Air Blown over Pipes That Contain a Flow of Chilled Water.

beled airflow, produces an estimate for param-eter Rdct, the air resistance in the duct. It per-forms this estimate using the following esti-mator for Fext,

Fext = (ρlkg + ρdmpr(Xdmpr)) ÏPdwctw /wRdwctw ,

which is derived using three equations in theoffice model pertaining to airflow. This actiondoes not take any parameter values as input(ρlkg and ρdmpr(Xdmpr) are known a priori as con-stants).

Consider a commonsense account of howthe estimation plan is generated. MORIARTY

constructs this plan by generating a set ofpossible estimation actions from the model.It then selects and sequences a subset of theactions sufficient to cover all parameters. Ac-tions are generated in two steps: In the firststep, subsets of the model and sensors areidentified that are sufficiently constrained toform an estimation problem. For example,the airflow action described earlier was gener-ated from the pressure Pdct and airflow sensorsF, together with the following airflow equa-tions:

Fext = Flkg + Fdmpr .

Flkg = 1 2ÏPwdcwtw .

Fdmpr = 1 2ÏPwdcwtw .ρdmpr(Xdmpr) }}

Rdct

ρlkg }Rdct

consisting of 14 equations that involve 17state variables and 11 parameters. About one-third of the equations are nonlinear, such as

Fdmpr = 1 2 ÏPwdcwtw ,

which relates airflow through the damper toduct pressure and duct air resistance as afunction of damper position. Nine of thestate variables are sensed, including tempera-ture T, flow rate F, air pressure P, and damperand reheat valve position X. Seven of the 11parameter values are unknown and must beestimated by MORIARTY.

MORIARTY’s task is to generate an estimationplan from the thermal model. MORIARTY, whenbrought to full capability, will be able to gen-erate the plan in figure 4 (it currently gener-ates less efficient plans). In terms of the im-mobot metaphor, each octagon in the figurerepresents an adaptive, autonomic estimationprocess, and the plan graph represents high-er-level coordination. Each octagon in the di-agram is an estimation action that is definedby an estimator y = f(x,p) applied to a subsetof the sensor data. The estimator is construct-ed from a subset of the model. The arc goinginto the action specifies additional parame-ters whose values are already known, and thearc leaving the action specifies parameterswhose values have been determined by theestimation. For example, the top octagon, la-

ρdmpr (Xdmpr) }}

Rdct

Articles

26 AI MAGAZINE

Figure 4. Model-Estimation Plan for a Single Office.

Air-Flw

Rht-Std Rht-Trn

Rm-Nght-Std

Rm-Nght-Trn

Rm-Day-Trn

R dct

Q rhtmax C rht

Rwall , Qeqp

Crm

Qslr

Qeqp

pRwall

MORIARTY’stask is to generate anestimationplan from the thermalmodel.

Two additional actions, Rht and Rm, are creat-ed from the set of temperature and heat-flowequations by using the duct air-temperaturesensor to split the set into reheat and roomsubmodels. The first action, Air-Flw, involvesthe single parameter Rdct, action Rht involvesthe two parameters Qrhtmax and Crht, and Rminvolves the four parameters Rwall, Qeqp, Crm,and Qslr(t). MORIARTY’s first step generates eightpossible estimation actions in total, each con-taining between one and seven parameters.Air-Flw, Rht, and Rm are three actions thatcover all seven model parameters and containthe fewest parameters individually. The factthat the sets of parameters in these actions aredisjoint is purely coincidental.

An additional step, under development,produces one or more simplified versions ofeach estimation action by identifying condi-tions on the data such that influences by oneor more parameters become negligible. Forexample, consider the estimator for actionRm:

= +

.

This estimator can be simplified by noticingthat solar effect Qslr(t) is negligible at night-time when the sun is down (action Rm-Nght), but it is significant during the day (ac-tion Rm-Day-Trn). Action Rm-Nght isgenerated from Rm by restricting the data setto data taken at night, which allows Qslr to beeliminated, reducing the number of parame-ters in the estimator from four to three: Rwall,Qeqp, and Crm.

Furthermore, the effect of the room heatcapacity Crm is negligible when the room tem-perature is in steady state (Rm-Nght-Std), butit becomes particularly significant duringroom-temperature transients (Rm-Nght-Trn).Hence, action Rm-Nght-Std is generated fromthe simplified action Rm-Nght by further re-stricting the data set to data where dTrm/dt issmall. Thus, the term CrmdTrm/dt can be elimi-nated from the estimator, and the number ofparameters can be reduced from three to two.Applying this simplification process to thefirst three of the eight actions generated inthe first step results in the six primitive esti-mation actions shown in figure 4.

Having generated these estimation actions,sequencing exploits the fact that some ofthese estimation actions share parameters tofurther reduce the dimensionality of thesearch performed by each action. In particu-lar, the output of an estimation action with

Qeqp + Qslr(t) + Rwall(Text – Trm)}}}}

Crm

C0Fsply (Tsply – Trm)}}}

Crm

dTrm}dt

fewer unknown parameters, such as Rm-Nght-Std, is passed to an overlapping actionwith more parameters, such as Rm-Nght. Se-quencing reduces the number of unknownparameters estimated in the second action;for example, Rm-Nght is left only with Crm asan unknown. This approach produces the se-quence shown in figure 4. Each action esti-mates at most two unknown parameters, adramatic reduction from the seven unknownparameters in the original problem.

Technically, in the first step, MORIARTY de-composes a model into an initial set of esti-mation actions by exploiting an analogy tothe way in which model-based diagnosis de-composes and solves large-scale multiple-faultproblems. The decomposition of a diagnosticproblem is based on the concept of aconflict—a minimal subset of a model (typi-cally in propositional or first-order logic) thatis inconsistent with the set of observations(de Kleer and Williams 1987; Reiter 1987).MORIARTY’s decompositional learning methodis based on the analogous concept of adissent—a minimal subset of an algebraicmodel that is overdetermined given a set ofsensed variables (that is, a dissent is justsufficient to induce an error function). Theset of three flow equations used earlier toconstruct the Air-Flw estimator is an exampleof a dissent. Following this analogy, MORIARTY

uses a dissent-generation algorithm (Williamsand Millar 1996) that parallels the conflict-recognition phase of model-based diagnosis.

MORIARTY’s simplification step (currently un-der development) is based on an order-of-magnitude simplification method called cari-catural modeling (Williams and Raiman 1994).Caricatural modeling partitions a model intoa set of operating regions in which some be-haviors dominate, and others become negligi-ble. MORIARTY applies caricatures to each esti-mation action generated in the first step,using the action’s estimator as the model.Each operating region generated correspondsto a simplified estimation action; the operat-ing region’s model is the simplified estimator,and the operating region’s boundary descrip-tion provides a filter on data points for the es-timator. In the thermal example, one ofmany simplifications to our estimator for Rmis to minimize Qslr(t). Because Qslr(t) ≈ 0 for alldata at night, this simplification can be used.In the final step, sequencing selects and or-ders primitive estimation actions using an al-gorithm that greedily minimizes the un-known parameters in each estimator, asdescribed in Williams and Millar (1996).

Consider MORIARTY’s performance on the ex-

Articles

FALL 1996 27

able. For example, at trial size 200, the origi-nal estimator requires 166 seconds, but thetotal time to estimate all parameters using F2,F1, and F6 is under 9 seconds, representing aspeedup by a factor of 14.

In addition, the sequence requires less datato converge, important for self-modeling im-mobile robots, which use online estimationto quickly track time-varying parameters.Based on the rule of thumb that the data-setsize should roughly be tenfold the dimensionof the parameter space, F7 would requirearound 70 data points, but the F2, F1, F6 se-quence requires only 40. The anomalousslowdown in the convergence rate of F7 at 25data points is attributed to insufficient data.Finally, although not included, parameter ac-curacy, measured by the confidence intervalof each parameter, is also improved using thegenerated sequence.

To summarize, a model-based approach isessential for regulating systems of the size of

ample, where the simplification step hasn’tbeen performed (figure 5). MORIARTY generatesthe eight estimators, F1 to F8, in the first stepand immediately sequences them to produce<F2, F1, F6>, which corresponds to the flowreheat and room-estimation actions, givenearlier. We compare the performance of the 8estimators by running them against sensordata sets ranging in size from 10 to 200,shown previously. The y axis denotes thetime required to converge on a final estimateof parameters involved in the dissent. Theplot labeled F7 is for the original seven-di-mensional estimator, and plots F2, F1, and F6are the three estimators in MORIARTY’s se-quence. Higher-dimensional estimators, suchas F7, tend to fail to converge given arbitraryinitial conditions; hence, ball-park initial pa-rameter estimates were supplied to allow con-vergence in the higher-dimensional cases. De-composition leads to significant speedupeven when good initial estimates were avail-

Articles

28 AI MAGAZINE

• • • ••

• •

•

Trial size

Cov

erge

nce

Tim

e (s

econ

ds)

0 50 100 150 200

050

100

150

•

•

•

•

••

•

•

• • • • • • • •• •• • • •

• •

• • • ••

•

••

• •• •

• •• •

• • • • • • • •• • • • • • • •

F8

F7

F6

F5

F4

F3

F2F1

Figure 5. Convergence Rate versus Data Size for the Generated Estimators F1 to F8.

most immobile robots. Embodying an immo-bile robot with self-modeling capabilities re-quires the use of symbolic reasoning to coor-dinate a large set of autonomic estimationprocesses. This coordination mimics the wayin which a community of modelers decom-poses, simplifies, and coordinates modelingproblems on a grand-challenge scale. DMLautomates one aspect of this rich model de-composition and analysis planning process.

The New Millennium Program HAL in 2001 highlights four aspects of the im-mune system of an immobile robot that weexamine technically in this section: First is theability to monitor internal health, detectingand isolating failing functions, processes, andcomponents. Second is the ability to continu-ally reconfigure the low-level autonomic pro-cesses of an immobot, establishing intendedfunctions and working around failures in acost-efficient and reliable manner. Third is theability to reason extensively throughreconfiguration options yet ensure reactivity.Fourth is the ability to reason about hybridsystems.

LivingstoneWe have developed a system called LIVING-STONE3 to investigate these issues in the con-text of NASA’s New Millennium Program. LIV-INGSTONE is a fast, reactive, model-basedconfiguration manager. Using a hierarchicalcontrol metaphor, LIVINGSTONE sits at thenexus between the high-level feed-forwardreasoning of classical planning-schedulingsystems and the low-level feedback responseof continuous adaptive control methods, pro-viding a kernel for model-based autonomy.LIVINGSTONE is distinguished from more tradi-tional robotic executives through the use ofdeliberative reasoning in the reactive feed-back loop. This deliberative reasoning is com-positional and model based, can entertain anenormous search space of feasible solutions,yet is extremely efficient because of the abili-ty to quickly focus on the few solutions thatare near optimal.

Three technical features of LIVINGSTONE areparticularly worth highlighting: First, the ap-proach unifies the dichotomy within AI be-tween deduction and reactivity (Brooks 1991;Agre and Chapman 1987). We achieve a reac-tive system that performs significant deduc-tion in the sense-response loop by drawingon our past experience at building fast propo-sitional conflict-based algorithms for model-based diagnosis and framing a model-basedconfiguration manager as a propositional

feedback controller that generates focused,optimal responses. Second, LIVINGSTONE’s rep-resentation formalism achieves broad cover-age of hybrid discrete-continuous, software-hardware systems by coupling the concurrenttransition-system models underlying concur-rent reactive languages (Manna and Pnueli1992) with the qualitative representations de-veloped in model-based reasoning. Third, thelong-held vision of model-based reasoninghas been to use a single central model to sup-port a diversity of engineering tasks. For mod-el-based autonomous systems, it means usinga single model to support a variety of execu-tion tasks, including tracking planner goals,confirming hardware modes, reconfiguringhardware, detecting anomalies, isolatingfaults, diagnosing, recovering from faults,and safing. LIVINGSTONE automates all thesetasks using a single model and a single-corealgorithm, thus making significant progresstoward achieving the model-based vision.

Configuration Management To understand the role of a configurationmanager, consider figure 6. It shows a sim-plified schematic of the main engine subsys-tem of Cassini, the most complex spacecraftbuilt to date. It consists of a helium tank, a fu-el tank, an oxidizer tank, a pair of main en-gines, regulators, latch valves, pyro valves,and pipes. The helium tank pressurizes thetwo propellant tanks, with the regulators act-ing to reduce the high helium pressure to alower working pressure. When propellantpaths to a main engine are open, the pressur-ized tanks force fuel and oxidizer into themain engine, where they combine and spon-taneously ignite, producing thrust. The pyrovalves can be fired exactly once; that is, theycan change state exactly once, either fromopen to closed or vice versa. Their function isto isolate parts of the main engine subsystemuntil needed or isolate failed parts. The latchvalves are controlled using valve drivers (notshown), and an accelerometer (not shown)senses the thrust generated by the main en-gines.

To start with the configuration shown inthe figure, the high-level goal of producingthrust can be achieved using a variety of dif-ferent configurations: Thrust can be provid-ed by either main engine, and there are anumber of different ways of opening propel-lant paths to either main engine. For exam-ple, thrust can be provided by opening thelatch valves leading to the engine on the leftor firing a pair of pyros and opening a set oflatch valves leading to the engine on the

Articles

FALL 1996 29

valves (rather than firing additional pyrovalves).

A configuration manager constantly at-tempts to move the spacecraft into lowest-cost configurations that achieve a set of high-level dynamically changing goals, such as thegoal of providing nominal thrust. When thespacecraft strays from the chosen configura-tion because of failures, the configurationmanager analyzes sensor data to identify thecurrent configuration of the spacecraft andthen moves the spacecraft to a new configura-tion, which, once again, achieves the desiredconfiguration goals. In this sense, a configura-tion manager such as LIVINGSTONE is a discretecontrol system that is strategically situated be-tween high-level planning and low-level con-trol; it ensures that the spacecraft’s configura-tion always achieves the set point defined bythe configuration goals.

Model-Based Configuration Management LIVINGSTONE is a reactive configuration manag-er that uses a compositional, component-based model of the spacecraft to determineconfiguration actions (figure 7). Each compo-nent is modeled as a transition system thatspecifies the behaviors of operating and fail-ure modes of the component, nominal andfailure transitions between modes, and thecosts and likelihoods of transitions (figure 8).Mode behaviors are specified using formulasin propositional logic, but transitions betweenmodes are specified using formulas in a re-stricted temporal, propositional logic. The re-stricted propositional, temporal logic is ade-quate for modeling digital hardware, analoghardware using qualitative abstractions (deKleer and Williams 1991; Weld and de Kleer1990), and real-time software using the mod-els of concurrent reactive systems in Mannaand Pnueli (1992). The spacecraft transition-system model is a composition of its compo-nent transition systems in which the set ofconfigurations of the spacecraft is the cross-product of the sets of component modes. Weassume that the component transition sys-tems operate synchronously; that is, for eachspacecraft transition, every component per-forms a transition.

A model-based configuration manager usesits transition-system model to both identifythe current configuration of the spacecraft,called mode identification (MI), and move thespacecraft into a new configuration thatachieves the desired configuration goals,called mode reconfiguration (MR). MI incre-mentally generates all spacecraft transitions

Articles

30 AI MAGAZINE

Figure 6. Schematic of the Cassini Main-Engine Subsystem.In the valve configuration shown, the engine on the left is firing.

Figure 7. Model-Based Reactive Configuration Management.

Heliumtank

Propellanttanks

Mainengines

Valve

Pyro valve

Regulator

Legend

MI MR

PlannerExecutive

ConfigurationgoalsConfirmation

CommandsObservations

ConfigurationManager

High-levelgoals

Model

right. Other configurations cor-respond to various combina-tions of pyro firings. The differ-ent configurations havedifferent characteristics becausepyro firings are irreversible ac-tions and because firing pyrovalves requires significantlymore power than opening orclosing latch valves.

Suppose that the main-enginesubsystem has been configuredto provide thrust from the leftmain engine by opening the

latch valves leading to it. Sup-pose that this engine fails, forexample, by overheating, so thatit fails to provide the desiredthrust. To ensure that the desiredthrust is provided even in thissituation, the spacecraft must betransitioned to a new configura-tion in which thrust is now pro-vided by the main engine on theright. Ideally, thrust is providedby firing the two pyro valvesleading to the right side andopening the remaining latch

from the previous configuration such that themodels of the resulting configurations areconsistent with the current observations(figure 9). MR determines the commands tobe sent to the spacecraft such that the result-ing transitions put the spacecraft into aconfiguration that achieves the configurationgoal in the next state (figure 10). The use of aspacecraft model in both MI and MR ensuresthat configuration goals are achieved correct-ly.

Both MI and MR are reactive. MI infers thecurrent configuration from knowledge of theprevious configuration and current observa-tions. MR only considers commands thatachieve the configuration goal in the nextstate. Given these commitments, the decisionto model component transitions as syn-chronous is key. An alternative is to modelmultiple component transitions through in-terleaving. However, such interleaving canplace an arbitrary distance between the cur-rent configuration and a goal configuration,defeating the desire to limit inference to asmall fixed number of states. Hence, we mod-el component transitions as being syn-chronous. If component transitions in theunderlying hardware-software are not syn-chronous, our modeling assumption is stillcorrect as long as some interleaving of transi-tions achieves the desired configuration.

In practice, MI and MR need not generateall transitions and control commands, respec-tively. Rather, just the most likely transitionsand an optimal control command are re-quired. We efficiently generate these by re-casting MI and MR as combinatorial opti-mization problems. In this reformulation, MIincrementally tracks the likely spacecraft tra-jectories by always extending the trajectoriesleading to the current configurations by themost likely transitions. MR then identifies thecommand with the lowest expected cost thattransitions from the likely current configura-tions to a configuration that achieves the de-sired goal. We efficiently solve these combi-natorial optimization problems using aconflict-directed best-first search algorithm.See Williams and Nayak (1996) for a formalcharacterization of MI and MR and a descrip-tion of the search algorithm.

A Quick Trip to SaturnFollowing the announcement of the NewMillennium Program in early 1995, space-craft engineers from the Jet Propulsion Labo-ratory (JPL) challenged a group of AI re-searchers at NASA Ames and JPL todemonstrate, within the short span of five

months, a fully autonomous architecture forspacecraft control. To evaluate the architec-ture, the JPL engineers defined the NEWMAAP

spacecraft and scenario based on Cassini.The NEWMAAP spacecraft is a scaled-down ver-sion of Cassini that retains the most chal-lenging aspects of spacecraft control. TheNEWMAAP scenario is based on the most com-plex mission phase of Cassini—successful in-sertion into Saturn’s orbit even in the eventof any single point of failure.

The AI researchers, working closely with thespacecraft engineers, developed an au-tonomous agent architecture that integratesLIVINGSTONE with the HSTS planning andscheduling system (Muscettola 1994) and amultithreaded smart executive (Pell, Gat, et al.1996) based on RAPS (Firby 1995). In this archi-tecture (see Pell, Bernard, et al. [1996] for de-tails), HSTS translates high-level goals into par-tially ordered tokens on resource time lines.The executive executes planner tokens bytranslating them into low-level spacecraft com-mands while enforcing temporal constraintsbetween tokens. LIVINGSTONE tracks spacecraftstate and planner tokens and reconfigures forfailed tokens. This autonomous agent architec-ture was demonstrated to successfully navigatethe simulated NEWMAAP spacecraft into Saturnorbit during its one-hour insertion window, de-spite about a half-dozen failures. Consequently,LIVINGSTONE, HSTS, and the smart executive havebeen selected to fly Deep Space One, formingthe core autonomy architecture of NASA’s NewMillennium Program.

Articles

FALL 1996 31

Figure 8. Transition-System Model of a Valve.Open and closed are normal operating modes, but stuck open and stuck closedare failure modes. The Open command has unit cost and causes a mode transi-tion from closed to open, similarly for the Close command. Failure transitionsmove the valve from the normal operating modes to one of the failure modes

with probability 0.01.

Open1

Close1

0.01

0.01

0.01

0.01

Open

Closed Stuck closed

Stuck open

Nominal transitionwith cost

Failure transitionwith probability

Articles

32 AI MAGAZINE

Previous configuration

Possible currentconfigurations with

observation "no thrust"

Possible nextconfigurations that

provide "nominal thrust"

Current configuration

Figure 10. Mode Reconfiguration (MR).The figure shows a situation in which mode identification (MI) has identified a failed main-engine valve leading to the left main engine.

MR reasons that normal thrust can be restored in the next state if an appropriate set of valves leading to the right engine are opened.The figure shows two of the many configurations that achieve the desired goal (circled valves are commanded to change state). Transi-tioning to the configuration at the top has lower cost because only necessary pyro valves are fired. The valves leading to the left engine

are turned off to satisfy a constraint that, at most, one engine can fire at one time.

Figure 9. Mode Identification (MI).The figure shows a situation in which the left engine is firing normally in the previous state, but no thrust is observed in the current

state. MI’s task is to identify the configurations into which the spacecraft has transitioned that account for this observation. The figureshows two possible transitions, corresponding to one of the main-engine valves failing “stuck closed” (failed valves are circled). Many

other transitions, including more unlikely double faults, can also account for the observations.

Table 1 provides summary informationabout LIVINGSTONE’s model of the NEWMAAP

spacecraft, demonstrating its complexity. TheNEWMAAP demonstration included seven fail-ure scenarios. From LIVINGSTONE’s viewpoint,each scenario required identifying likely fail-ure transitions using MI and deciding on a setof control commands to recover from the fail-ure using MR. Table 2 shows the results ofrunning LIVINGSTONE on these scenarios.

The first column in table 2 names each ofthe scenarios, but a discussion of the details ofthese scenarios is beyond the scope of this arti-cle. The second and fifth columns show thenumber of solutions checked by MI and MR,respectively. One can see that even though thespacecraft model is large, the use of conflict-di-rected search dramatically focuses the search.The third column shows the number of lead-ing trajectory extensions identified by MI. Thelimited sensing available on the NEWMAAP

spacecraft often makes it impossible to identi-fy unique trajectories. The fourth and sixthcolumns show the time spent by MI and MRon each scenario, once again demonstratingthe efficiency of our approach.

The New MillenniumWe are only now becoming aware of therapid construction of a ubiquitous, immobilerobot infrastructure that rivals the construc-tion of the World Wide Web and has the po-tential for profound social, economic, andenvironmental change. Tapping into this po-tential will require embodying immobotswith sophisticated regulatory and immunesystems that accurately and robustly controltheir complex internal functions. Developingthese systems requires fundamental advancesin model-based autonomous system architec-tures that are self-modeling, self-configuring,and model based programmable and supportdeliberated reactions. This development canonly be accomplished through a coupling ofthe diverse set of high-level, symbolic meth-ods and adaptive autonomic methods offeredby AI. Although a mere glimmer of Spock andHAL, our two model-based immobots, LIVING-STONE and MORIARTY, provide seeds for an ex-citing new millennium.

Acknowledgments We would like to thank the many individualswho have influenced this work. Oren Et-zioni’s provocative 1992 AAAI Spring Sympo-sium talk on softbots provided a catalyst.Thanks to the responsive environment team,particularly Bill Millar, Joseph O’Sullivan, and

Ying Zhang. Thanks to our remote-agent col-laborators, including James Kurien, NicolaMuscettola, Barney Pell, Greg Swietek, Dou-glas Bernard, Steve Chien, Erann Gat, andReid Simmons. Finally, thanks to YvonneClearwater for help with the graphics.

Notes1. This article is based on an invited talk given atthe Third International Conference on AI PlanningSystems (Williams 1996).

2. To achieve tractability, a modeler of nonlinearphysical systems typically searches for estimatorscontaining four or less parameters.

3. LIVINGSTONE the program is named after DavidLivingstone (1813–1873), the nineteenth-centurymedical missionary and explorer. Like David Liv-ingstone, LIVINGSTONE the program is concernedwith exploration and the health of explorers.

ReferencesAgre, P., and Chapman, D. 1987. PENGI: An Imple-mentation of a Theory of Activity. In Proceedingsof the Sixth National Conference on Artificial Intel-ligence, 268–272. Menlo Park, Calif.: American As-sociation for Artificial Intelligence.

Brooks, R. A. 1991. Intelligence without Reason. InProceedings of the Twelfth International Joint Con-ference on Artificial Intelligence, 569–595. Menlo

Articles

FALL 1996 33

Number of components 80Average number of modes per component 3.5Number of propositions 3,424Number of clauses 11,101

Scenario MI MRCheck Accept Time Check Time

EGA preaim failure 7 2 2.2 4 1.7BPLVD failed 5 2 2.7 8 2.9IRU failed 4 2 1.5 4 1.6EGA burn failure 7 2 2.2 11 3.6Acc failed 4 2 2.5 5 1.9ME too hot 6 2 2.4 13 3.8Acc low 16 3 5.5 20 6.1

Table 1. NEWMAAP Spacecraft Model Properties.

Table 2. Results from the Seven NEWMAAP Failure-Recovery Scenarios.Time is in seconds on a SPARC 5.

and M. Zweben, 169–212. San Francisco, Calif.:Morgan Kaufmann.

Pell, B.; Gat, E.; Keesing, R.; and Muscettola, N.1996. Plan Execution for Autonomous Spacecraft.Paper presented at the AAAI Fall Symposium onPlan Execution, 9–11 November, Cambridge, Mas-sachusetts.

Pell, B.; Bernard, D. E.; Chien, S. A.; Gat, E.;Muscettola, N.; Nayak, P. P.; Wagner, M. D.; andWilliams, B. C. 1996. A Remote-Agent Prototype forSpacecraft Autonomy. In Proceedings of the SPIEConference on Optical Science, Engineering, andInstrumentation, Volume on Space SciencecraftControl and Tracking in the New Millennium.Bellingham, Wash.: Society of Professional ImageEngineers.

Rasmussen, R. 1996. Personal communication, Na-tional Aeronautics and Space Administration,Washington, D.C.

Reiter, R. 1987. A Theory of Diagnosis from FirstPrinciples. Artificial Intelligence 32(1): 57–96.

Sacks, E. P., and Doyle, J. 1992. Prolegomena toAny Future Qualitative Physics. Computational In-telligence 8(2): 187–209.

Weld, D. S., and de Kleer, J., eds. 1990. Readings inQualitative Reasoning about Physical Systems. SanFrancisco, Calif.: Morgan Kaufmann.

Williams, B. C. 1996. Model-Based AutonomousSystems in the New Millennium. In Proceedings ofthe Third International Conference on Artificial Intelli-gence Planning Systems, 275–282. Menlo Park, Calif.:AAAI Press.

Williams, B. C. 1990. Interaction-Based Invention:Designing Novel Devices from First Principles. InProceedings of the Eighth National Conference onArtificial Intelligence, 349–356. Menlo Park, Calif.:American Association for Artificial Intelligence.

Williams, B. C., and Millar, B. 1996. Automated De-composition of Model-Based Learning Problems. InProceedings of the Tenth International Workshopon Qualitative Reasoning, 265–273. AAAI TechnicalReport WS-96-01. Menlo Park, Calif.: American As-sociation for Artificial Intelligence.

Williams, B. C., and Nayak, P. P. 1996. A Model-Based Approach to Reactive Self-Configuring Sys-tems. In Proceedings of the Thirteenth NationalConference on Artificial Intelligence, 971–978.Menlo Park, Calif.: American Association forArtificial Intelligence.

Williams, B. C., and Raiman, O. 1994. Decomposi-tional Modeling through Caricatural Reasoning. InProceedings of the Twelfth National Conference onArtificial Intelligence, 1199–1204. Menlo Park,Calif.: American Association for Artificial Intelli-gence.

Zhang, Y.; Williams, B. C.; and Elrod, S. 1994. Mod-el Estimation and Energy-Efficient Control forBuilding Management Systems. SPL Technical Re-port, Xerox PARC, Palo Alto, California.

Park, Calif.: International Joint Conferences on Ar-tificial Intelligence.

de Kleer, J., and Williams, B. C., eds. 1991. ArtificialIntelligence, Volume 51. New York: Elsevier.

de Kleer, J., and Williams, B. C. 1987. DiagnosingMultiple Faults. Artificial Intelligence 32(1): 97–130.

Dent, L.; Boticario, J.; McDermott, J.; Mitchell, T.;and Zabowski, D. 1992. A Personal Learning Ap-prentice. In Proceedings of the Tenth NationalConference on Artificial Intelligence, 96–103. Men-lo Park, Calif.: American Association for ArtificialIntelligence.

Elrod, S.; Hall, G.; Costana, R.; Dixon, M.; and desRivieres, J. 1993. Responsive Office Environments.Communications of the ACM 36(7): 84–85.

Etzioni, O., and Segal, R. 1992. Softbots as Testbedsfor Machine Learning. Paper Presented at 1992AAAI Spring Symposium on Knowledge Assimila-tion, 25–27 March, Stanford, California.

Etzioni, O., and Weld, D. 1994. A Softbot-Based In-terface to the Internet. Communications of the ACM37(7): 72–79.

Firby, R. J. 1995. The RAP Language Manual. Ani-mate Agent Project Working Note, AAP-6, Universi-ty of Chicago.

Hamscher, W. C. 1991. Modeling Digital Circuitsfor Troubleshooting. Artificial Intelligence51:223–271.

Kautz, H.; Selman, B.; Coen, M.; Ketchpel, S.; andRamming, C. 1994. An Experiment in the Design ofSoftware Agents. In Proceedings of the Twelfth Na-tional Conference on Artificial Intelligence,438–443. Menlo Park, Calif.: American Associationfor Artificial Intelligence.

Knoblock, C. A., and Levy, A. Y., eds. 1995. Paperpresented at the AAAI Spring Symposium Series onInformation Gathering in Distributed and Hetero-geneous Environments, 27–29 March, Stanford,California.

Levy, A. Y.; Rajaraman, A.; and Ordille, J. J. 1996.Query-Answering Algorithms for InformationAgents. In Proceedings of the Thirteenth NationalConference on Artificial Intelligence, 40–47. MenloPark, Calif.: American Association for Artificial In-telligence.

Maes, P., and Kozierok, R. 1993. Learning InterfaceAgents. In Proceedings of the Eleventh NationalConference on Artificial Intelligence, 459–465.Menlo Park, Calif.: American Association forArtificial Intelligence.

Malik, A., and Struss, P. 1996. Diagnosis of Dynam-ic Systems Does Not Necessarily Require Simula-tion. In Proceedings of the Tenth InternationalWorkshop on Qualitative Reasoning, 127–136.AAAI Technical Report WS-96-01. Menlo Park,Calif.: American Association for Artificial Intelli-gence.

Manna, Z., and Pnueli, A. 1992. The Temporal Logicof Reactive and Concurrent Systems: Specification. NewYork: Springer-Verlag.

Muscettola, N. 1994. HSTS: Integrating Planning andScheduling. In Intelligent Scheduling, eds. M. Fox

Articles

34 AI MAGAZINE

Brian Williams is a research sci-entist in the Computational Sci-ences Division of the NASA AmesResearch Center. He is co-lead ofthe Model-Based AutonomousSystems Project and a flight leadfor the New Millennium Deep-Space One spacecraft. He receivedhis professional degrees from the

Massachusetts Institute of Technology—an SB inelectrical engineering in 1981, and SM in computerscience in 1984, and a Ph.D. in 1989. He was amember of the technical staff at Xerox PARC until1994. His current research concentrates on the de-velopment and formalization of model-based au-tonomous systems for controlling immobile robots,including deep-space probes and smart buildings.Research interests include model-based informationgathering, ranging from diagnosis to large-scale ex-ploratory modeling and data analysis, hybrid algo-rithms that use symbolic reasoning to coordinateadaptive methods, and formal theories of abstrac-tion and approximation.

JAI Press Ad

Articles

FALL 1996 35

Pandurang Nayak is a researchscientist in the ComputationalSciences Division of the NASAAmes Research Center. He is co-lead of the Model-Based Auto-nomous Systems Project and amember of the New MillenniumDeep-Space One flight softwareteam. He received a B.Tech. in

computer science and engineering from the IndianInstitute of Technology, Bombay, in 1985, and aPh.D. in computer science from Stanford Universi-ty in 1992. His research interests include abstrac-tions and approximations in knowledge represen-tation and reasoning, the building of model-basedautonomous systems, diagnosis and recovery, andqualitative and causal reasoning.