In this article we describe a concept for a new type of mate-rial, which we call claytronics (Goldstein, Campbell, and Mowry2005), made out of very large numbers—potentially millions—of submillimeter-sized spherical robots. While still only a con-cept, we have completed a considerable amount of initial designand experimentation work, enough at this point to allow us tounderstand what is readily achievable within a short time frame(less than a decade) and also to identify some of the most sig-nificant technical challenges yet to be overcome. To date, wehave developed and analyzed several promising engineeringdesigns, conducted numerous large-scale experiments on ahigh-fidelity physics-based simulator, and successfully carriedout several prototype three-dimensional (3D) microelectro-mechanical systems (MEMS) manufacturing runs. These experi-ences lead us to believe that there are no fundamental softwareor hardware barriers to realizing claytronics on a large scale andwithin a few years.

While the most fundamental purpose of our research onclaytronics is to understand manufacturing and programmingof very large ensembles of independently actuated computingdevices, it is also clear that such a material would have numer-ous practical applications, ranging from shape-shifting radioantennas (important for software-defined radios) to 3D faxmachines. Perhaps our most fanciful-sounding application,however, is motivated by one of the most basic of human needs:to communicate and interact with others. Two centuries ago,the only practical way to carry on a real-time conversation with

Articles

SUMMER 2009 29Copyright © 2009, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Beyond Audio and Video: Using Claytronics to

Enable Pario

Seth C. Goldstein, Todd C. Mowry, Jason D. Campbell, Michael P. Ashley-Rollman, Michael De Rosa, Stanislav Funiak, James F. Hoburg, Mustafa E. Karagozler, Brian Kirby, Peter Lee,

Padmanabhan Pillai, J. Robert Reid, Daniel D. Stancil, and Michael P. Weller

n In this article, we describe the hardware andsoftware challenges involved in realizingclaytronics, a form of programmable mattermade out of very large numbers—potentiallymillions—of submillimeter-sized sphericalrobots. The goal of the Claytronics Project is tocreate ensembles of cooperating submillimeterrobots, which work together to form dynamicthree-dimensional physical objects. For exam-ple, claytronics might be used in telepresense tomimic, with high-fidelity and in three-dimen-sional solid form, the look, feel, and motion ofthe person at the other end of the telephone call.To achieve this long-range vision we are inves-tigating hardware mechanisms for constructingsubmillimeter robots, which can be manufac-tured en masse using photolithography. We alsopropose the creation of a new media type, whichwe call pario. The idea behind pario is to renderarbitrary moving, physical three-dimensionalobjects that you can see, touch, and even holdin your hands. In parallel with our hardwareeffort, we are developing novel distributed pro-gramming languages and algorithms to controlthe ensembles, LDP and Meld. Pario may fun-damentally change how we communicate withothers and interact with the world around us.Our research results to date suggest that there isa viable path to implementing both the hard-ware and software necessary for claytronics,which is a form of programmable matter thatcan be used to implement pario. While we havemade significant progress, there is still muchresearch ahead in order to turn this vision intoreality.

another person was to meet face-to-face, requiringboth people to be in the same place at the sametime. The invention of the telephone had a pro-found impact on society, and our ability to con-verse with people who are far away is somethingthat we take for granted. Still, despite several gen-erations of experience, using the telephone (or,today, the videoconference) is still not the same asmeeting with someone face to face.

Hence we propose the creation of a new mediatype, which we call pario.1 Similar to how audioand video allow us to render arbitrary sounds andmoving images over long distances, the ideabehind pario is to render arbitrary moving, physi-cal 3D objects that you can see, touch, and evenhold in your hands. As with audio and video,when we reproduce something with pario we areneither transporting the original object nor creat-ing an exact replica: instead, the idea is to create aphysical artifact that is a “good enough” reproduc-tion of the shape, appearance, and motion of theoriginal object—one that our senses will accept asbeing real.

As a concrete technology for pario, claytronicsserves as a programmable form of modeling claywhose shape and appearance are remotely control-lable, thereby creating physical objects that youcan directly see and touch.

While we must admit that the notion of pario isfanciful, the practical impact of realizing it wouldbe enormous. For example, consider how peoplecollaborate to design 3D objects today (such asarchitects designing a house, or engineers design-ing a car). A mathematical representation of the 3Dobject is captured within the computer using someform of CAD software. While the ability to edit andshare this representation is extremely powerful,how do the designers interact with this abstractmodel? Today, a monitor displays a (flat) perspec-tive on the object from a particular viewpoint. Tohelp imagine the object in three dimensions ratherthan two dimensions, a designer might spin theobject around on the screen to see different per-spectives. To modify the object, a designer reachesfor a mouse or keyboard. The awkwardness of thisinteraction suggests that this is a case where peopleare bending to the limitations of technology,rather than technology rising to the level of whatis most natural and useful to humans. While animprovement, virtual or augmented reality sys-tems still limit user interaction, requiring the userto don special equipment, limiting the number ofusers that can interact with the environment. Mostimportantly, they do not allow the user to actuallytouch the artifacts.

In these two examples, the fundamental causeof the gap between what people want and whattechnology delivers is that computation is limitedby its media types (for example, text, audio, video),

and therefore is confined to “cyberspace.” Thiseither limits what technology can deliver or forceshumans to adapt to technology in awkward ways.

Why Claytronics Is Well Suited toMeeting the Requirements of Pario

Taking pario from a fanciful vision to a practicalreality will be extremely challenging. In this sec-tion, we begin by describing the requirementsimposed by pario on any underlying technologythat might be used to implement it, given the goalof creating a seamless pario experience. Whilethere may be a number of different approaches toimplementing pario, we describe in this sectionwhy we believe that the particular technology thatwe are pursuing—that is, claytronics—is an excel-lent match for the needs of pario.

Requirements for ParioPario requires an underlying technology that canconstruct a wide range2 of high-fidelity dynamicphysical objects at a large enough scale and withsufficient structural stability to enable hands-onhuman interaction. Note that existing 3D outputtechnologies are insufficient for the following rea-sons. Although head-mounted displays or holo-grams can provide the illusion of a 3D object, nophysical interaction with the object is possible.3

While 3D printers (for example, fused depositionmodeling)4 have revolutionized the way thatdesign and engineering takes place today, they canonly render static objects, as opposed to the mov-ing, dynamic objects necessary for pario.

We now consider each aspect of the pariorequirements in greater detail to see how it trans-lates to specific properties and requirements forclaytronics. Later, in the claytronics hardware andprogramming sections, we describe our approachto realizing claytronics (including our currentimplementation status) in the context of meetingthese requirements.

How Claytronics Meets These RequirementsWe need to render a wide range of dynamic shapes,eliminating the possibility of implementing pariowith a monolithic device. A monolithic devicecould not, for example, seamlessly create shapesthat are not topologically equivalent. Hence thefirst requirement for claytronics is that it must bean ensemble of individual particles that worktogether to create the shape. We call each individ-ual unit of the ensemble a catom, short forclaytronic atom. Notice that the requirements ofpario do not demand a particular base technologyfor these units: they could be cells constructedusing synthetic biology or, in our case, catoms con-structed using silicon and photolithography. To

Articles

30 AI MAGAZINE

support dynamic shapes (as well as shape forma-tion), the catoms will need mechanisms for mov-ing around each other. To maintain a given shapeand to provide sufficient structural stability forphysical human interaction, the catoms will needto be able to adhere to each other with sufficientforce to maintain stability and exert force on oth-er objects in the world.

We now consider the implications of wantinghigh-fidelity objects that can be constructed on alarge enough scale for successful human interac-tion. Drawing an analogy to video technology, asuccessful video display must have sufficientlysmall, densely packed pixels to deliver a high-fidelity image, and the overall display size must belarge enough for comfortable viewing. For pario,the individual catoms can be thought of as physi-cal voxels (the physical 3D equivalent of pixels);we would like them to be small (preferably lessthan 1 millimeter in diameter) in order to deliverhigh 3D physical fidelity. To convey visual infor-mation, each catom needs to be able to change itscolor (similar to a pixel). If we also want to conveyhigh-fidelity tactile information (for example, doesan object feel like wood or glass when you run yourfinger across it), it should be possible to synthesizemany textures by spacing the catoms on the sur-face in the proper arrangement, provided they areless than 0.3 millimeters in diameter (Klatzky andLederman 2006, Lederman et al. 2006).

While high fidelity dictates submillimeter units,the requirement of constructing large enoughobjects for human interaction dictates that we willneed massive numbers of them: on the order ofmillions. This constraint limits the kinds of manu-facturing processes that can be used to build theunits to one that is at least as scalable and cost-effective as a bulk-manufacturing method such asphotolithography or synthetic biology. It alsoplaces constraints on the number of different kindsof elements that can make up the ensemble. We donot expect to achieve a seamless pario experiencein the near term and instead aim at creating anensemble of homogeneous submillimeter catomsthat would be capable (in one form or another) ofmeeting the pario requirements.

The scale of the ensemble requires that compu-tation be integrated into the ensemble. The maintask of an ensemble at any given time is to repro-duce a particular shape in three dimensions. Thus,at any given time each unit will have to execute aparticular and, in general, different task dependingon its current position. For example a unit mighthave to move to a new position, increase ordecrease its adhesion forces along a particular vec-tor, or change its color. The computation neededto compute these tasks is most easily located with-in the ensemble itself. This reduces the amount ofcommunication bandwidth necessary to control

the ensemble, and it increases the robustness of theensemble. Contrast catoms to pixels on a display.In the latter case centralized computation workswell. However, pixels only have state—they do notinteract with each other. Catoms, on the otherhand, need to interact with each other (such aswhen moving, sharing power, sensing the envi-ronment, and so on) in real time. If control wascentralized, or even spread across a few locations,then communication between the catoms and theprocessor would introduce latency. Even for medi-um-sized ensembles, such a delay in the controlloop would likely render the ensemble ineffective.Scaling to large ensembles would also becomeharder due to bandwidth constraints. Alternative-ly, consider the cells in our body: each one hassome local “processing” and can perform signifi-cant functions independently. Of course, the indi-vidual units will have to be able to communicatewith their neighbors and some must be able tocommunicate with external devices.

Another common feature of any technologythat implements pario is that the individual unitswill require energy. Luckily, this does not meanthat each unit must have its own independentsource of energy. Instead, there must be two mech-anisms to supply energy to a unit. At least someunits must be capable of receiving power from anexternal source and transmitting that power toother units in the ensemble. The rest of the unitsmust be able to receive (and transmit) power from(or to) their neighbors. To facilitate robustness andmomentary disconnectedness, it would be advan-tageous if each unit could store some energy.

In the long run it will most likely be advanta-geous for the ensemble to be made up of a small setof specialized units. For example, some might bespecialized for computation, others for externalcommunication or energy storage. The units mayeven have different shapes to support, for example,the building of complex mechanisms. In theClaytronics Project we have chosen to limit ourinvestigations in the near term to ensembles madeup of homogeneous units, thus significantly sim-plifying the programming problem as all the unitsare interchangeable.

Claytronics HardwareAt first glance the ability to create a coherentensemble of millions of units that meet the aboverequirements appears fantastical—almost sciencefiction. But, if we step back and examine it, thequestion is not if we can manufacture it, but when.It is clearly possible to do so in principle; for exam-ple, biology builds ensembles of units that coordi-nate together to form dynamic 3D shapes that caninteract in the real world, and we already have anentire industry based on bulk manufacturing

Articles

SUMMER 2009 31

today—the semiconductor industry has beenbuilding computer chips for decades using photo-lithography. The same basic process used to man-ufacture chips has also been used to commerciallycreate MEMS circuits that are integrated on thesame die with mechanical systems (Madou 2002).MEMS processes can also be used to create 3Ddevices. Reid has constructed spherical shapes byfirst printing a projection of the sphere and then,by harnessing the inherent stresses in thin film sil-icon dioxide, causing the projection to fold up intoa sphere (see figure 1) (Reid, Vasilyev, and Webster2008; Vasilyev, Reid, and Webster 2008). This sameprocess can be applied to a prefabricated CMOSwafer to create 3D units with integrated processorsand actuators.

As we hinted earlier, MEMS represent just onepossible implementation technology. As the indi-vidual units scale down in size, other technologiesthat can produce less expensive units will have tobe used. For example, researchers at Intel haveshown that the submillimeter spherical shells canbe formed using inexpensive materials in siliconmolds.5 In the long term, approaches based onsynthetic biology (Endy 2005) are very appealing.

Ensemble PrincipleIndependent of the implementation technologythere are some common requirements imposed byour desire to create ensembles of millions of coop-

erating units. Because our goals are orientedaround the ensemble as a whole, units only needto function when they are part of the ensemble.This leads to one of our guiding engineering prin-ciples, the ensemble principle: an individual unitshould include only enough functionality to con-tribute to the desired functionality of the ensem-ble. Keeping this principle in mind forces us tosimplify the hardware and software, ideally to thepoint where each unit is as simple as possible,enabling inexpensive robust units and thus morerobust ensembles. A concrete example of thisrelates to how the units move. Note that therequirements listed in the previous section do notrequire that the individual units move independ-ently. In fact, there is nothing that says that theindividual units even need to include a mecha-nism for locomotion. If an external force, forexample, air currents, is applied to the ensemble, itis possible in principle for the units to reform intoa new shape by controlling when and where theyadhere to each other. By letting go at one place andthen sticking at another they can form arbitraryshapes. A less extreme example would incorporatea mechanism that can be used to allow cooperat-ing units to move, but not allow an individual unitto move without cooperating with other units inthe ensemble. For example, in our early prototypesthe individual units used electromagnets for move-ment (Kirby et al. 2007). Units move when twoneighboring units each energize their magnet inthe appropriate direction. This also allowed us tosimplify the individual units by eliminating allmoving parts.

Basic Catom MechanismsIn the remainder of this section we describe onepossible path to realizing claytronics. The mecha-nisms discussed here are not intended to be finalsolutions, but rather to show that at least an earlyversion of pario can be realized using techniquesand methods that are understood today. After pre-senting our current approach we briefly discusssome of the more challenging aspects to buildinga catom.

Using the ensemble principle as a guide we showhow a monolithically manufactured unit can com-pute, communicate, move, adhere, sense, andshare power with other units using just two basicmechanisms: a processor and an arrangement ofconductive plates under the surface controlled bythe processor. As will soon be clear, the smaller thecatom, the easier it is to actuate. However, thecatom needs to be large enough for the processor,storage capacitor, and other circuits. For this dis-cussion we will assume that the diameter of thecatom is 0.7 millimeters. This gives us a totalusable area for circuits of just over 1.5 square mil-limeters.

Articles

32 AI MAGAZINE

Figure 1. Spherical Shell.

An example of a spherical shell (with diameter of approximate 0.9 millime-ters) made using standard photolithography and stress-induced curling. Theshell (in the upper left) is sitting on a circuit board that can move the shellusing electrostatic forces.

For the reasons given earlier, each unit mustcontain a processor and memory. Even using old-er 90 nanometer technology we could include acore similar to the ARM7,6 64 kilobytes of non-volatile memory and 64 kilobytes of double datarate dynamic random access memory (DRAM) in0.3 square millimeters. A clock rate of 30 kilohertzfor approximately 0.03 million of instructions persecond (MIPS) would be sufficient to run the pro-grams necessary for ensemble behavior and allowus to keep nominal power consumption at 1.8microwatts. Of course, as process technologyimproves we could reduce the size of the catom orincrease processing power or memory capacity andpotentially lower the energy requirements. Thenumbers and examples we cite here and in theremainder of the section are used only to showthat even with current technology it is possible tomeet the needs of a catom.

We use electrostatic forces for catom movement,adhesion, communication, sensing, and power dis-tribution. The electrostatic forces are generated byhaving the processor route charge to conductingplates that are printed on the die. These plates willbe just under the surface of the catom beneath adielectric layer of silicon dioxide (SiO2) (figure 2).The geometry of the catom ensures that when twocatoms are adjacent to each other they will createcapacitors between adjacent electrodes. Sincecatoms cannot share a common ground, catomsare coupled by using pairs of electrodes. When avoltage is applied between the plates of a capaci-tor, attractive forces are created due to the accu-mulated charge on the plates (see figure 2). Thissame basic mechanism is used for all the electro-static-based functions. Note that our use of elec-trostatics for adhesion, movement, sensing, andpower distribution is an example of the ensembleprinciple. The mechanism is very simple, yet nosingle catom can move, and so on, without inter-action with other catoms.

Motion of the whole ensemble occurs whenlarge numbers of individual catoms move them-selves around their neighbors. Individual catomsonly move by rolling around their neighbors intoa vacant space, either on the outer surface of theensemble, or into a void in the interior of theensemble. This limits the number of catoms thatcan move at any given time placing constraints onthe ensemble motion-control algorithms. Howev-er, algorithms such as those described in De Rosa etal. (2006) and Dewey et al. (2008) operate with justthis sort of behavior. This form of motion requiresthat actuation be sufficient to move only onecatom. Since the moving catom does not rubagainst any others, friction is negligible. When thecatom on top is to roll around another catomclockwise (see figure 3 for a profile of the move-ment), a voltage difference is applied between all

the plates that are located in the bottom rightquadrant of the upper catom and the electrodesthat are located at the upper right quadrant of thelower catom, as shown in figure 2.

The force generated by the electrostatic plates isa function of catom diameter, electrode size andspacing, and applied voltage. In designing a catomof a specified size (in this case 0.7 millimetersdiameter), the force generated at a fixed voltage isfirst calculated as a function of the number of elec-trodes resulting in the plot like that shown in fig-ure 4. Based on the plot, the optimal number ofelectrodes for a 0.7 millimeter diameter sphere is43, or roughly one electrode every 8–9 degrees.Using this electrode count, the voltage required tomove the catom vertically against gravity (assum-ing the catom has 1/13 the density of water) isapproximately 94 volts. This voltage decreaseswith the catom diameter because as the catomscales down in size, the torque required to moveagainst gravity decreases faster than the torquegenerated by the electrostatic force. For a catomwith a 0.5 millimeter diameter the required volt-age is 60 volts. This then suggests that the smallerthe catom, the better.

In the near term, practical considerations pushfor larger catom sizes. A catom diameter of 0.7 mil-limeters provides a nice compromise. The easiestway to move the catoms would be to have themmove one diameter and then come to a completerest before moving again. With the assumptionsmade above and assuming that a catom comes tocomplete rest after moving one diameter, catomsshould be able to move at least 20 body lengths a

Articles

SUMMER 2009 33

Figure 2. Electrodes Are Placed Radially on the Catom Surface.

second horizontally and at least 10 body lengths asecond against gravity. This is far short of thespeeds needed for generalized pario, but reasonablefor nearer-term applications. For faster operation,dynamics and drag need to be taken into account.

Electrostatics can also be used to cause adjacentcatoms to adhere to each other. In this case, theelectrodes on each catom are oppositely charged.Again, using the above assumptions, electrostatic-based adhesion should allow a catom to carry theweight of at least 10,000 catoms hanging from it,or a chain of catoms 8 meters long. Of course, sucha structure would be incredibly frail. In practicelarge structures will have many layers that com-bine to resist external forces and the like.

Power can be shared between catoms using thesame mechanism. Much as a transformer uses anelectromagnetic coupling between coils to trans-mit power across an insulator, a pair of capacitive-ly coupled electrodes can be used to carry energyacross a nonconducting gap. Figure 5 shows thebasic idea—an alternating current (AC) signal isput on the plates on the catom with power, andthe capacitive coupling causes mirror charges toappear on the adjacent catom. This signal is recti-fied and stored in a local storage capacitor. Analy-sis similar to that described by Karagozler et al.(2007) shows that using this mechanism, powercan be transferred with between 50 percent and 66percent efficiency in less than 40 milliseconds fora millimeter-scale catom. The lack of efficiency of

this mechanism requires us to explore alternativemeans of providing power to the vast majority ofcatoms. Among the mechanisms we are exploringis providing power through transmitted energysuch as radio frequency (RF) or optical. In the RFspace we are looking at magnetic resonant cou-pling (Cannon et al. 2008). In the optical space weare examining small solar cells that can fit into thecatom. Such a cell with cross-sectional area of 0.25square millimeters can provide 15.7 microwatts(solar mass is equal to 1.71, 8.3 percent efficiency).Conservatively, our baseline design is done with a10 microwatt power budget.

A simple low-power communication mecha-nism would modulate the voltage on the actuatorplate adjacent to the neighboring catom. The mod-ulation is accomplished indirectly by charging anddischarging a smaller plate embedded below themain actuation plate. This isolates the smallerplate from high voltage on the main plates. It alsohas a negligible effect on the adhesion forces need-ed to maintain ensemble cohesion. Driving thecommunication plates will capacitively propagatethe signal to the neighbor’s plates. The energyrequired to communicate in this way is, in theworst case where we dump the charge after everybit, about 0.7 picojoules per bit. Thus, even a 1megabit channel would require less than 1microwatt. This method uses very little power butis susceptible to noise that may arise from mechan-ical vibration of the catoms. A more power-inten-

Articles

34 AI MAGAZINE

D

Figure 3: The Motion Profile of a Catom Moving along the Ensemble.

sive communication method would modulate acarrier that would add an additional 1.7 micro -watts.

The communication plates can also be used tosense the presence of neighbors. Additionally, ifthe surface of the catom has any conformance,increased pressure between adjacent catoms will bedetectable as an increase in capacitance. If thisproves insufficient we can embed piezoelectricstrain sensors in the silicon shell of each catom. Itshould be noted that in either case, no singlecatom will be able to accurately determine thedirection and amount of force being applied.Instead, they will have to communicate with eachother to effectively create a synthetic sensor.

The final major piece necessary for even an ear-ly version of pario is for each voxel to be able tochange its color. Cognizant of the power require-ments of a display, and to give claytronics a morereal appearance, we have elected to pursue a reflec-tive approach. Each catom will be multicolored.The catom will rotate in place to present the prop-er color to the user. This approach means that nopower is required to maintain a particular color.

The plan outlined here requires a nominal pow-er budget of less than 10 microwatts. Commercial-ly available EIA size 0201 capacitors7 would provideenough energy to run for about 250 milliseconds.At this power level the unit can main tain a reason-

able operating temperature through black bodyradiation. If we use capacitive coupling for powersharing, this means that in the worst case thecatom spends more than 80 percent of its timedoing useful work.

Significant ChallengesThe design sketch presented above shows that allthe functionality required to implement claytron-ics, at least for medium-scale ensembles, is possi-ble. However, there remain many engineeringchallenges before catoms are a reality. Table 1 givesa brief outline of the path we are taking towardimplementing claytronics along with the status ofthe individual components.

As expected, the processing and communicationhardware is sufficient for even the most challeng-ing applications. Furthermore, as semiconductorfabrication continues to improve we will haveeven more resources capable of satisfying the com-puting needs of claytronics. Supercapacitor tech-nology continues to improve, but even today’scommercially available packages appear to providesufficient storage capacity. The display mechanismalso appears to be sufficient, given small enoughcatoms. Finally, the electrostatic-based actuationmechanism, combined with the right control soft-ware, will give the catoms sufficient capability andspeed for movement.

Articles

SUMMER 2009 35

Number of Electrodes

Torq

ue (

10

–1

0 N

m)

2.8

2.6

2.4

2.2

2

1.8

1.6

1.410 20 30 40 50 60 70 80 90 100

Figure 4. Generated Torque Versus Number of Uniformly Distributed Electrodes for a 0.7 Millimeter Diameter Catom at 100 Volt Bias.

From a hardware point of view, the two keychallenges are power distribution and adhesion.Capacitive coupling, which is a good starting pointfor both, is not sufficient to scale to large ensem-bles. In the case of power distribution, capacitivecoupling is not efficient enough to transmit powerthrough a long chain of catoms. We are currentlyinvestigating several alternatives including photo-voltaics and magnetic resonant coupling (Kurs etal. 2007, Cannon et al. 2008). In both of these cas-es, the catoms can be essentially made transparent.Thus, only the most central units—the ones withthe least to do—will require capacitive coupling forpower. For adhesion, a straightforward applicationof capacitive coupling is impressive and goodenough for many near-term pario applications, butit will not be sufficient to create the kinds of arbi-trary structures necessary for generalized pario.Solving this problem will likely require drawing onreversible chemistries, that is, chemical processthat involve the reversible formation of chemicalbonds without side reactions.

Claytronics, as we have described it here, ismade up of a homogeneous ensemble of units. Webelieve that this is the proper place to start explor-ing programmable matter, but not the endingplace. If rendered objects are going to move in anatural way, catoms cannot just flow around eachother. Instead, they will need to assemble into

components that have some relationship to theobject they are rendering. In the case of a human,this would involve creating joints, bones, and mus-cles. In all likelihood this will require several typesof units in the ensemble.

Programming ClaytronicsThere are significant challenges in programmingany distributed system. Programmable matter pos-es additional challenges that arise from limitedresources, an ever-changing topology of networkconnections that result from the units movingaround, and high action-uncertainty as a result ofthe constant interaction with the physical world.The programming challenges can broadly be divid-ed into two areas: programming the individualunits and programming the ensemble.

The issues involved in programming an individ-ual unit are essentially the same as those related toprogramming individual robots, particularly mod-ular robots (Yim et al. 2007) (that can be viewed asan early form of programmable matter). The criti-cal difference between modular robotics and theclaytronics vision is that of scale: we must consid-er many more much smaller robotic modules. Verysmall units pose challenges due to lesser capabili-ties, in terms of programming resources, noisiersensing, and lesser physical actuation capability.

Articles

36 AI MAGAZINE

Rectificationand

Regulation

Capacitive coupling

Moduleconnectedto agroup ofpoweredmodules

Module being powered

Processingand

Electronics

DC

Figure 5. Scheme for Transferring Power between Catoms.

Our main focus has been on the problem of pro-gramming the ensemble as a whole. One of themost challenging aspects of programming ensem-bles with several orders of magnitude more unitsthan typically considered in modular robotics is tofind and express scalable algorithms. One of themain impediments to this, as in any massively dis-

tributed systems, is that the programmer mustwrite code for each node in the system individual-ly, and yet ensure the overall distributed systemreaches the correct high-level result.

Traditional imperative programming languages,such as C, C++, and Java, do little to addressensemble-level programming issues. These lan-

Articles

SUMMER 2009 37

Capability Status Description

Individual Actuated Unit Physical Structure

Demo-mm Basic stress-induced sphere creation demonstrated (Reid et al. 2008). (Other methods also demonstrated5 or in the design phase.)

Energy Collection

Demo-cm Demonstration of capacitive-based mechanisms at mac-roscale (Karagozler et al. 2007, Karagozler 2007); Design and analysis of photonic based collection; Demonstra-tion of cm-scale resonant coupling mechanism (Cannon et al. 2008)

Energy Storage Commercial Capacitors of the right volume and capacity are com-mercially available.

Actuation Demo-mm Demonstrated at the mm-scale (Karagozler et al. 20079).

Adhesion Demo Electrostatic-based adhesion demonstrated at the mac-roscale and mm-scale (Karagozler et al. 20079); Fiber-based adhesion demonstrated at centimeters and below (Murphy et al. 2007a, 2007b).

Integration In Process Tests integrating the above are being carried out.

Programmable Actuated Unit Processing Commercial Commercially available cores will fit in area and energy

budget with sufficient processing requirements.

Memory Commercial As above.

External communication (input only)

Design The capability to recieve data from the outside for pro-gramming and control of the units.

Small Ensembles Contact sensing Design Using changes in capacitance on surface.

Neighbor-to-neighbor communication

Design Using capacative coupling as described above.

Large Ensembles Energy Transfer Demo-cm Transfer of power using capacitive coupling demon-

strated at the cm-scale (Karagozler et al. 2007, Karagozler 2007).

Generalized Pario-capable Ensembles Color Design As described above.

Specialization Ideas Specialized units for power storage, external communi-cation, and possible mechanical structural.

Table 1: Current Status of the Mechanisms Necessary and the Design Path for Claytronics.

Status key: Commercial: Already available commercially; Demo-mm: Demonstrated at the mm scale; Demo=cm:Demonstrated at the cm scale; Design: Paper designs, simulations, and analysis—yet to be demonstrated; In Process:Currently working on demonstrating; Idea: Still working on the design.

guages are inherently oriented towards a singleprocessing node and require significant additionaleffort when used in a distributed setting. In addi-tion to creating a representation of the data need-ed for an algorithm, the programmer must deter-mine what information is available locally andwhat must be obtained from remote nodes, themessages and protocol used to transfer this data,mechanisms to route or propagate informationthrough multiple hops as needed, and a means toensure the consistency of this data. Furthermore,in algorithms to control ensembles, it is almostalways necessary to express the conditions andactions that need to be carried out in terms ofinformation that spans multiple units. Languagesthat constrain the programmer to the perspectiveof a single node make such algorithms difficult toimplement.

Successful approaches to this programmingproblem must in the end deal with but one issue,scalability. The first condition required to ensurescalability is to allow the programmer to think ofthe ensemble as an ensemble, and not as manyindividual nodes. This allows the programmer tofocus on the overall problem, not the details of theindividual nodes. We push onto the compiler thetask of compiling the ensemble-level descriptioninto low-level code for each node. Next, the lan-guage should support a concise, but understand-able, expression of the algorithm. We use programlength as a metric because shorter programs aregenerally easier to understand, modify, debug, andoptimize. Scalability also requires uncertainty tol-erance to handle the failures inherent in a systemwith large numbers of nodes working in the phys-ical world. Finally, to cope with the exponentialnumber of states and transitions in such a system,the language and tools should support formalmethods for proving the program is correct.

One path to programming ensembles is toexplore emergent approaches such as cellularautomata. This is seductive because often a smallset of simple rules can result in interesting andcomplex behavior. However, despite significanteffort in this area, there is still no automatic way ofdetermining, in general, the behavior that willarise in an ensemble from the local rules of theagents. Instead, we take the opposite approach, tospecify high-level global behavior that is compileddown into local programs to be run on the units ofthe ensemble. While such an engineered approachmay not derive the smallest set of rules necessary,there are hints (based on recent research) that it ispossible. This is particularly true in the case wherepeople have investigated programming methodsto render shapes in massively distributed systemsof interacting agents (Jones and Matariae 2004;Zykov and Lipson 2006; Klavins, Ghrist, and Lip-sky 2006; De Rosa et al. 2006; Abelson et al. 2000;

Nagpal 2001; and Stoy and Nagpal 2004). Thus, webelieve that pario—whose main goal is to render3D objects—is a good place to start to understandthe problem of programming programmable mat-ter.

Ensemble AlgorithmsAll of the activity in claytronics revolves aroundthe ensemble as a whole. In keeping with theensemble axiom, the individual units are very sim-ple and not fully capable of functioning on theirown. Rather, a group of units or even the entireensemble needs to work in concert for many essen-tial functions. Even to bootstrap and power up aclaytronics ensemble requires collaboration. Anensemble placed on a powered surface (such as agrid of power and ground lines) will have access topower only on its surface. We have developed apartially directed, semirandomized algorithm(Campbell, Pillai, and Goldstein 2005) that allowsa few powered units to generate and extend a pow-er grid throughout the ensemble, eventually dis-tributing power to all the units. We have leveragedemergent behavior in our approach, since the unitsstart without any power and are initially able torun only very simple programs.

Once the nodes are all powered, they can runmore complex code. The functions and actionsthat must be performed by each unit for pario areoften determined by the position of the unit in theensemble. Hence, the next task is for the units toestablish an ensemble-wide coordinate system.They do this by sensing where their neighbors arerelative to their own coordinate frame and by run-ning a distributed algorithm that determines aconsistent coordinate system across the ensemble,even when the sensor readings are very noisy. Ouralgorithm for this internal localization task per-forms global pose estimation of the units by hier-archically partitioning the ensemble into smallercomponents, recursively applying the algorithm tothe smaller pieces, applying rigid alignment tomerge partial solutions, and using distributed, iter-ative refinement to smooth the results. This algo-rithm has been shown to determine the locationsof the catoms in large ensembles to within 3 per-cent of their true position, when each module has12 sensors around its perimeter (Funiak et al.2008). Furthermore, it is highly scalable, that is,the communication complexity of the algorithmscales logarithmically in the number of units. Inaddition to providing power and performing inter-nal localization, the ensemble may at this pointundertake other startup procedures, such as estab-lishing a network overlay for efficient communica-tion, or constructing a picture of the environmentby combining the sensed information from multi-ple surface units.

At this point the ensemble is ready to morph

Articles

38 AI MAGAZINE

into a particular shape. Our current approach toensemble shape change uses a generalized meta-module planner (Dewey et al. 2008) that operateson subcollections of modules (also called meta-modules) that have fewer motion constraints thanthe individual modules. This allows for efficient,distributed shape planning, with a lower-level con-troller converting metamodule motion primitivesinto a sequence of module motions. In our system,a 3D model of the desired shape is broadcast to theensemble and then distributed among the catoms.Each of the catoms then executes a small distrib-uted program that attempts to create or destroymetamodules to achieve the goal shape. In coordi-nation with local neighbors, the catoms move toeffect the creation and deletion of metamodules,thus morphing the ensemble to the target shape.Although our algorithm is nondeterministic, itsimplementation has been proven to be both soundand complete: barring failures, the program willnot disconnect the ensemble, and if the targetshape can be reached, then it will eventually bereached.

These three ensemble tasks illustrate some of thedifferent approaches one can take in creating pro-grams for an ensemble. The power distribution andshape change use emergent techniques, and relyon a few simple rules with local sensing and com-munication to achieve a fairly complex task. Ingeneral, it is very difficult to determine a set oflocal rules to actually cause a desired ensemble-lev-el behavior, but once such rules are constructed,they are easily implemented and executed on the

individual nodes. The localization program, on theother hand, uses a more top-down approach, withan algorithm developed independent of the localoperations of the modules and then translated torun on the ensemble. Implementation of this algo-rithm is much more involved, requiring significantcross-ensemble coordination, communication,and synchronization to work effectively. On theother hand, this approach allowed us to effect thedesired complex, multistage behavior across theensemble.

Programming Languages for ClaytronicsIn pursuit of our goal we are exploring differentprogramming approaches and have developed twonew programming languages: LDP (De Rosa et al.2008, Ashley-Rollman et al. 2007a) and Meld (Ash-ley-Rollman et al. 2007b). Both of these languagesare declarative in nature and result in programsthat are about 20 times shorter than equivalentimperative programs. They each take an ensembleperspective, allowing a programmer to create sim-ple, concise programs that are automatically com-piled down to programs that run on each unit. Inthe examples shown in figures 6 and 7, each lan-guage is used to implement a simple three-nodelocomotion algorithm, whose steps are illustratedin figure 8.

Meld is a declarative logic programming languagebased on Datalog (Ceri, Gottlob, and Tanca 1989)and inspired by P2 (Loo et al. 2006). A Meld pro-gram consists of facts and the rules for derivingthem. A fact represents the current program state

Articles

SUMMER 2009 39

// Calculate the distance to the targetdist(Catom,Distance):- at(Catom,CatomLocation),

TargetLocation = destination(), // Retreive the destinationDistance = |CatomLocation - TargetLocation|, // Determine distance to destination for CatomDistance > robot-radius. // Stopping condition, that is, Catom at destination

// Determine which module is the farthest awayfarther(FarCatom,NearCatom):- neighbor(FarCatom,NearCatom),

dist(FarCatom,DistanceF arCatom),dist(NearCatom,DistanceNearCatom),DistanceFarCatom ≥ DistanceNearCatom.

// Move MovingCatom around PivotCatom until it touches TargetCatommoveAround(MovingCatom,PivotCatom,TargetCatom):- farther(MovingCatom,PivotCatom),

farther(MovingCatom,TargetCatom),TargetCatom �= PivotCatom.

Figure 6. Walk Program in Meld.

including observations about the world, actionsthat should be performed, the goal or result of analgorithm, and any internal algorithm state. Thefacts representing observations and those that per-form an action are used to interface the ensemblewith the environment. The observation factsinclude sensor data as well as information aboutthe network topology of the catoms. The networktopology information (neighbor facts in the exam-ple in figure 6) is especially important to both Meldand the programmer. Meld uses these facts to auto-matically distribute data throughout the ensemble.The programmer needs these facts in order to effec-tively understand the physical geometry of theensemble. The action facts activate an actuatorwhen derived; for example, moveAround causesthe catom to rotate around a neighbor.

A Meld program, such as the one shown in fig-ure 6, is executed through a process called bottom-up reasoning or forward-chaining. The observationsabout the world (neighbor and at) constitute thestarting set of known facts. The rules of the pro-gram are matched against these facts to derive newfacts that are then added to the set of known facts.When an observed fact changes it is removed fromthe set of known facts through a process calleddeletion. Deletion continues until all the derivedfacts based on the old deleted fact are also deleted.This allows the program state always to reflect thecurrent state of the world, simplifying coding forthe programmer and providing an automaticmeans for discovery of and recovery from failures.The programmer only needs to specify the high-level logical relations and reasoning rules; thecompiler takes care of the low-level operations:efficient communication of facts between theunits, application of the rules, ensuring forwardprogress on proofs, and deletion of derivationswhen facts are refuted or deleted. The programmer

is freed from the burden of determining what mes-sages to send and to what units, and from manag-ing of data at each node. Algorithms that requirevery long, complex programs in C or C++ style lan-guages can be expressed in just a few lines in Meld.

Locally distributed predicates (LDP) is a declara-tive programming language derived from distrib-uted watchpoints (De Rosa et al. 2007), a debug-ging facility designed to identify and detectmultinode error conditions in ensembles. LDPallows programmers to specify distributed condi-tions among small, connected groups of nodes,and to trigger actions when groups matching thecondition are detected. LDP treats each node as acollection of named state variables and allows forthe expression of predicates that combine nodestate, historical state, and topological constraints.The underlying LDP run time uses radius-limiteddistributed snapshots to continuously search formatching groups of nodes. Once a predicatematches it can trigger arbitrary actions. Theseactions can include modifying state variables, call-ing arbitrary C functions, and rearranging thenode’s local topology through movement. ThoughLDP provides fewer proof properties than Meld,the LDP run time is designed to integrate easilywith lower-level C code, providing (for instance)access to sensor readings directly as state variables.With the exception of such low-level interactions,all variable storage and messaging is handled bythe LDP run time, allowing for extremely conciseexpression of distributed algorithms.

Current State of the ArtBoth Meld and LDP have been shown (Ashley-Roll-man et al. 2007b, De Rosa et al. 2008, Ashley-Roll-man et al. 2007a) to create programs that are sig-nificantly smaller (approximately 20 times) thantheir imperative counterparts. For example, a com-

Articles

40 AI MAGAZINE

scalar radius;point location,destLocation;scalar distance = INT_MAX;

#update the distance from a to destination, if a is not already thereforall(a) where (a.distance > a.radius) do

a.distance = |a.location - destLocation|;#if a is the farthest catom, it rotates around c until it contacts bforall(a, b, c) where (a.distance > b.distance) & (a.distance ≥ c.distance) do

a.moveAround(c.id,b.id);

Figure 7. Walk Program in LDP.

plete shape planner, as described in Dewey et al.(2008) is implemented in just 28 lines of LDP code.

One of the advantages of concise programs isthat it is possible for the programmer to thinkabout the whole program, which facilitates cor-rectness and optimizations. We see this in the per-formance of the implementations. ComparingMeld and C++ to implement applications hasshown that the Meld compiler is effective in dis-tributing the application among the nodes (Ash-ley-Rollman et al. 2007b). In the case of larger pro-grams we often see better performance in the Meldimplementation. The main reason for this is thatthe Meld implementation makes use of much ofthe latent parallelism inherent in the algorithm,while the C++ implementation is limited to theparallelism that the programmer can manage andexplicitly encodes.

Our investigations into fault tolerance and auto-matic verification of programs is still in its infancy.Preliminary results indicate that the ensemble-cen-tered programming styles embodied in LDP andMeld lead directly to fault-tolerant programs. InMeld, for example, when a unit fails to move to thecorrect location, the system automatically removesany derivations that do not agree with the currentlocation of the unit. We have shown that runningour shape planning algorithm on one million unitsstill results in success even when more than 10 per-cent of the units fail during execution.

In the area of automatic verification, we havebeen able to prove important properties of our pro-grams using a combination of paper proofs and astraightforward manual translation from Meldsource to inputs for theorem-proving tools likeTwelf (Pfenning and Schurmann 1999). This per-mits us to prove correctness properties about algo-rithms and their actual implementations. Forexample, we have proven that our general shape-

planning algorithm (Dewey et al. 2008) written inMeld is complete (that is, if the target shape isreachable, it will be created) and sound (that is, theensemble will never become disconnected).

Research ChallengesSignificant research challenges remain in themethods and languages used to program program-mable matter. Both of the declarative languages wehave developed treat the ensemble as a set of net-worked processing nodes. This has worked well forthe relatively low-level tasks we have thus farinvestigated, but it is not clear if this is the rightabstraction for higher-level tasks. For many parioapplications, it may be desirable to have direct lan-guage support (for example, for spatial informa-tion and temporal dynamics) to describe the high-level shapes and motions desired. This may requireabstractions similar to those used in 3D graphicsprogramming or modeling. Whether a completelynew language is needed for this or if these abstrac-tions can be built on top of our existing declarativelanguages (such as with the metamodule abstrac-tion used for shape change) remains an open ques-tion.

Pario Application ScenariosHaving described some of the technology behindclaytronics that might be used to implement pario,we now present an application scenario to illus-trate how pario might be used in practice at somepoint in the future. The following scenario willappear, at first glance, to be pure fantasy; however,as we described earlier, there is a path toward real-izing a technology (claytronics) that will enablepario.

When Dr. Alice Smith saw the extent of the chest

Articles

SUMMER 2009 41

a b

c

a b

c a

b

c a

cb

c a

cb

a

a b c d e

Figure 8. Illustration of Catoms Collectively “Walking.”

a. Starting location at origin; b. Catom a moves around b; c. Catom a finishes moving; d. Catom c now moves around a; e. Catom c finishesmoving.

injury to Fred, who had just been rushed into theemergency room, she knew that she wanted to havean immediate face-to-face meeting with two of hercolleagues: Bob and Charlie. At this moment, Bob isat home, and Charlie is working in a hospital in adifferent city. Fortunately, Bob’s house and bothhospitals have rooms that are equipped for “pario-conferencing,” which means that they contain rel-atively little furniture, a sandboxlike container fullof claytronics, and an array of cameras discreetlyplaced in the walls and ceiling for performing 3Dcapture. Alice initiates the meeting by sending elec-tronic invitations to Bob and Charlie through theInternet (similar to how one initiates a video chatsession today). Once Bob and Charlie accept theseinvitations, a digital connection is establishedbetween the three rooms that will be used to keepthem synchronized.

The camera array in Alice’s room captures theshape and appearance of Alice and any furniture orother objects in her room, and a correspondingprocess occurs in Bob’s and Charlie’s rooms. Aftersoftware confirms that it is safe to do so (for example,there won’t be any collisions), the claytronics mate-rial in Bob’s room forms into the shapes of Alice andthe chair that she is sitting on, and colors itself totake on their respective appearances. The same thinghappens in the reverse direction: a copy of Bob andhis furniture are rendered by means of claytronics inAlice’s room. Finally, both Alice and Bob are ren-dered in Charlie’s room, and Charlie is rendered inboth of their rooms. The net effect is that each roomwill contain the same sets of physical objects, wheresome objects are the originals and others are parioreplicas from other rooms. As people or objects inany of the rooms move, their replicas also moveaccordingly as they are continuously updated basedupon the input from the camera arrays.

The performance requirements of this “face-to-face” meeting scenario are perhaps the most chal-lenging of any pario application (given the desirefor realistic motion and real-time responsiveness ofhumanoid objects); hence we expect this aspect ofthe application scenario to have the longest timehorizon until it is practical. Our current design forclaytronics will not support this scale of an ensem-ble with this level of fidelity, for example. As thescenario continues, we illustrate what we expect tobe nearer-term applications of pario and claytron-ics.

Now that the “face-to-face” meeting has begun,Alice, Bob, and Charlie get to work. Using MRI orultrasound, the 3D structure of Fred’s injured chesthas been scanned. While Fred is resting in a hospi-tal bed in yet another room, a pario model of hischest is rendered using more claytronics material ineach of the three rooms. The three doctors interac-tively explore this physical model to understandthe injury: they “peel away” layers on the outside ofthe model to expose the internal organs, they“zoom in” or “zoom out” (that is, cause the modelto grow or shrink), they watch the flow of blood(highlighted by various colors in the claytronicsmaterial) through the circulatory system in thechest. While this is happening, the doctors are talk-ing to each other, looking at each other and eachchange to the model of Fred’s chest is synchronizedacross the rooms: it is as though they are standingin the same room, interacting with the same phys-ical 3D model of the patient.

The interactive 3D physical model applicationwe just described has a different set of require-ments and benefits. Unlike the face-to-face exam-ple, the claytronics material does not need to be

Articles

42 AI MAGAZINE

Figure 9. A Trumpet Being Formed.

Some snapshots of a trumpet being formed from an ensemble of one million catoms running a generalized metamodule planner (Deweyet al. 2008) on our simulator.8

other AI applications involving verylarge numbers of cooperating nodes.

AcknowledgementsThis research was sponsored in part bythe National Science Foundationunder grant no. CNS-0428738, IntelResearch Pittsburgh, Carnegie MellonUniversity, and Microsoft Research.The authors would like to thank thereviewers for their excellent com-ments.

Notes1. From the Latin root par, “to make, to cre-ate.”

2. Ideally, we would like to construct arbi-trary shapes, but this is too stringent arequirement given that the individual unitsof the ensemble are not made of the sameatoms as the object they are rendering. Forexample, they may not be able to form ashape made from titanium, since the bondsbetween titanium atoms will allow formoment arms that will exceed those thatcan be formed between the units.

3. Haptic input devices do not provide thefull range of physical interactivity neces-sary for pario applications.

4. See Stratasys, www.stratasys.com.

5. For more information, see the webcastby Justin Rattner “Research and Develop-ment: Crossing the Chasm betweenHumans and Machines,” Day 3 Keynote,Thursday, August 21, 2008, on Intel’s web-site (www.intel.com/pressroom/kits/events/idffall_2008/video.htm).

6. See www.arm.com/products/CPUs/ARM7TDMIS.html.

7. See Panasonic’s web page, MultilayerCeramic Capacitors (for general usage),specifically Ecjzeb0j224m. (industrial.pana-sonic.com/www-data/pdf/ABJ0000/ABJ0000CE1.pdf).

8. See www.pittsburgh.intel-research.net/dprweb/.

9. See also Milimeter Scale Catoms (www.cs.cmu.edu/~claytronics/hardware/millis-cale.html).

ReferencesAbelson, H.; Weiss, R.; Coore, D. A. D.;Hanson, C.; Homsy, G.; Thomas F.; Knight,J.; Nagpal, R.; Rauch, E.; and Sussman, G. J.2000. Amorphous Computing. Communi-cations of the ACM 43(4): 74–82 (www.swiss.ai.mit.edu/projects/amorphous/).

Ashley-Rollman, M. P.; De Rosa, M.; Srini-vasa, S. S.; Pillai, P.; Goldstein, S. C.; andCampbell, J. D. 2007a. Declarative Program-ming for Modular Robots. Paper presented at

mobile, requires lower adhesionforces, and can move significantlyslower, and has fewer constraints onwhat acceptable motion will berequired.

The initial geometry and appear-ance of the object (in this case, Fred’schest) do not necessarily come from acamera array, but may come from oth-er arbitrary sources; while we usedMRI or ultrasound as the source of the3D geometry in this example, theinformation might also come from aCAD program, a scientific computingsimulation, and so on. (See figure 9 foran example of how we currently, insimulation, render 3D objects.) Toenable hands-on interaction with thephysical model, the claytronics hard-ware must be able to detect touch orpressure to understand how the user ismanipulating the object. Unlike atouch-screen device, where only theimage on the screen can change inresponse to human touch, the shapeand other physical properties of theobject (for example, strength of resis-tive forces, tactile properties, and soon) can change in an application-spe-cific way: such as “peeling away” par-ticular layers of tissue or bone (bymoving them out of the way) in thecase of Fred’s chest model. Interactive3D models are compelling in theirown right, and they may be used with-out the face-to-face meeting aspect ofthe scenario.

The doctors realize that they alsowant to consult with Diane (a heartsurgeon), who is currently hiking on aremote trail in the woods. BecauseDiane is in a remote area and shepacked light, she does not haveenough claytronics material with herfor full-blown pario-conferencing.However, Diane does have a smallamount of claytronics with her in herpocket (it is currently shaped like acellphone). Alice, Bob, and Charliecall Diane; they upload the model ofFred’s beating heart into her claytron-ics, which renders itself into a scaled-down version of the same model thatAlice, Bob, and Charlie are currentlyholding in their hands. As Dianemanipulates her model with herhands, the same changes are immedi-ately reflected in the other models,and the doctors agree on their treat-ment plan for Fred. Some of thesemanipulations are even captured

directly by software and later used toguide the actual operation on Fred.

This final aspect of the scenarioillustrates how the fact that claytron-ics can morph between shapes (forexample, from a cellphone to a scaled-down model of a patient’s chest) maybe very useful in a mobile environ-ment. There are many variations onthis basic idea, including objects thatmorph between a shape that is con-venient to carry and a shape with alarge display, antennas that changetheir own shape to dynamically adaptthe transmission and reception prop-erties, and perhaps even small ingest-ed medical devices that can adapt howthey sense and actuate as they movethrough the digestive tract. In theselatter cases, useful devices might beconstructed at smaller physical scales,requiring fewer catoms (that is, at ascale of an ensemble that is compati-ble with our current claytronicsdesign).

ConclusionsAs the application scenarios illustrate,pario may fundamentally change howwe communicate with others andinteract with the world around us. Ourresearch results to date suggest thatthere is a viable path to implementingboth the hardware and software nec-essary for claytronics, which is a formof programmable matter that can beused to implement pario. While wehave made significant progress, thereis still much research ahead in order toturn this vision into reality.

While the realization of pario wouldenable exciting new applications, wealso believe that continued researchon claytronics itself may offer sidebenefits to fields such as robotics andmachine learning because of scale: weenvision millions of (tiny) robotsworking together to perform a singletask (that is, physically rendering anarbitrary moving 3D object), and thisis well beyond the scale of typical dis-tributed robotics applications. Hencethe techniques that will need to bedeveloped for effectively sensing, rea-soning, planning, and actuating onthis very large scale to make claytron-ics and pario work may also be benefi-cial to future distributed robotics or

Articles

SUMMER 2009 43

the Workshop on Self-ReconfigurableRobots/Systems and Applications at IROS’07, Friday, 2 November, San Diego, CA.

Ashley-Rollman, M. P.; Goldstein, S. C.;Lee, P.; Mowry, T. C.; and Pillai, P. 2007b.Meld: A Declarative Approach to Program-ming Ensembles. In Proceedings of theIEEE/RSJ International Conference on Intelli-gent Robots and Systems (IROS ’07). Piscat-away, NJ: Institute of Electrical and Elec-tronics Engineers, Inc.

Campbell, J. D.; Pillai, P.; and Goldstein, S.C. 2005. The Robot Is the Tether: Active,Adaptive Power Routing for ModularRobots with Unary Inter-Robot Connec-tors, 4108–15. In Proceedings of the IEEE/RSJInternational Conference on Intelligent Robotsand Systems (IROS ’05). Piscataway, NJ: Insti-tute of Electrical and Electronics Engineers,Inc.

Cannon, B.; Hoburg, J.; Stancil, D.; andGoldstein, S. 2008. Magnetic ResonantCoupling as a Potential Means for WirelessPower Transfer to Multiple Small Receivers.Technical Report CMU-CS-08-167,Carnegie Mellon University, Pittsburgh, PA.

Ceri, S.; Gottlob, G.; and Tanca, L. 1989.What You Always Wanted to Know aboutDatalog (and Never Dared to Ask). IEEETransactions on Knowledge and Data Engi-neering 1(1): 146–166.

De Rosa, M.; Goldstein, S. C.; Lee, P.; Camp-bell, J. D.; and Pillai, P. 2006. Scalable ShapeSculpting Via Hole Motion: Motion Plan-ning in Lattice-Constrained ModuleRobots. In Proceedings of the 2006 IEEE Inter-national Conference on Robotics and Automa-tion (ICRA ’06). Piscataway, NJ: Institute ofElectrical and Electronics Engineers, Inc.

De Rosa, M.; Goldstein, S. C.; Lee, P.; Camp-bell, J. D.; Pillai, P.; and Mowry, T. C. 2007.Distributed Watchpoints: Debugging LargeMultirobot Systems. In Proceedings of the2007 IEEE International Conference on Robot-ics and Automation (ICRA ’07). Piscataway,NJ: Institute of Electrical and ElectronicsEngineers, Inc.

De Rosa, M.; Goldstein, S. C.; Lee, P.; Camp-bell, J. D.; and Pillai, P. 2008. ProgrammingModular Robots with Locally DistributedPredicates. In Proceedings of the 2008 IEEEInternational Conference on Robotics andAutomation (ICRA ’08). Piscataway, NJ:Institute of Electrical and Electronics Engi-neers, Inc.

Dewey, D.; Srinivasa, S. S.; Ashley-Rollman,M. P.; De Rosa, M.; Pillai, P.; Mowry, T. C.;Campbell, J. D.; and Goldstein, S. C. 2008.Generalizing Metamodules to SimplifyPlanning in Modular Robotic Systems. InProceedings of the IEEE/RSJ International Con-ference on Intelligent Robots and Systems(IROS ’08). Piscataway, NJ: Institute of Elec-trical and Electronics Engineers, Inc.

Endy, D. 2005. Foundations for Engineer-ing Biology. Nature 438 (24 November):449–53.

Funiak, S.; Pillai, P.; Ashley-Rollman, M. P.;Campbell, J. D.; and Goldstein, S. C. 2008.Distributed Localization of Modular RobotEnsembles. In Proceedings of Robotics: Scienceand Systems. Cambridge, MA: The MITPress.

Goldstein, S. C.; Campbell, J. D.; andMowry, T. C. 2005. Programmable Matter.IEEE Computer 38(6): 99–101.

Jones, C., and Matariae, M. 2004. FromLocal to Global Behavior in Intelligent Self-Assembly. In Proceedings of the 2004 IEEEInternational Conference on Robotics andAutomation (ICRA ’04). Piscataway, NJ:Institute of Electrical and Electronics Engi-neers, Inc.

Karagozler, M. E. 2007. Harnessing Capaci-tance for Inter-Robot Latching, Communi-cation, and Power Transfer. Master’s thesis,Carnegie Mellon University Department ofElectrical and Computer Engineering. Pitts-burgh, PA.

Karagozler, M. E.; Campbell, J. D.; Fedder,G. K.; Goldstein, S. C.; Weller, M. P.; andYoon, B. W. 2007. Electrostatic Latchingfor Inter-Module Adhesion, Power Trans-fer, and Communication in ModularRobots. In Proceedings of the IEEE/RSJ Inter-national Conference on Intelligent Robots andSystems (IROS ’07). Piscataway, NJ: Insti-tute of Electrical and Electronics Engi-neers, Inc.

Kirby, B.; Aksak, B.; Goldstein, S. C.;Hoburg, J. F.; Mowry, T. C.; and Pillai, P.2007. A Modular Robotic System UsingMagnetic Force Effectors. In Proceedings ofthe IEEE/RSJ International Conference on Intel-ligent Robots and Systems (IROS ’07). Piscat-away, NJ: Institute of Electrical and Elec-tronics Engineers, Inc.

Klatzky, R. L., and Lederman, S. J. 2006.The Perceived Roughness of Resistive Vir-tual Textures: I. Rendering by a Force-Feed-back Mouse. ACM Transactions on AppliedPerceptron (TAP) 3(1): 1–14.

Klavins, E.; Ghrist, R.; and Lipsky, J. 2006.A Grammatical Approach to Self-Organiz-ing Robotic Systems. IEEE Transactions onAutomatic Control. 51(6): 949.

Kurs, A.; Karalis, A.; Moffatt, R.;Joannopoulos, J. D.; Fisher, P.; and Soljacic,M. 2007. Wireless Power Transfer ViaStrongly Coupled Magnetic Resonances.Science 317(5834): 83–6.

Lederman, S. J.; Klatzky, R. L.; Tong, C.; andHamilton, C. 2006. The Perceived Rough-ness of Resistive Virtual Textures: II. Effectsof Varying Viscosity with a Force-FeedbackDevice. ACM Transactions on Applied Percep-tron (TAP) 3(1): 15–30.

Loo, B.; Condie, T.; Garofalakis, M.; Gay,D.; Hellerstein, J. M.; Maniatis, P.; Ramakr-ishnan, R.; Roscoe, T.; and Stoica, I. 2006.Declarative Networking: Language, Execu-tion and Optimization. In Proceedings of the2006 ACM SIGMOD International Conferenceon Management of Data, 97–108. New York:ACM Press.

Madou, M. J. 2002. Fundamentals of Micro-fabrication: The Science of Miniaturization.Boca Raton, FL: CRC Press.

Murphy, M.; Aksak, B.; and Sitti, M. 2007a.Adhesion and Anisotropic FrictionEnhancements of Angled HeterogeneousMicro-Fiber Arrays with Spherical and Spat-ula Tips. Journal of Adhesion Science andTechnology 21(12–13): 1281–1296.

Murphy, M.; Aksak, B.; and Sitti, M. 2007b.Adhesion Behavior of Vertical and AlignedPolymer Microfibers. In Proceedings of Adhe-sion Society Symposium. Blacksburg, VA:Adhesion Society.

Nagpal, R. 2001. Programmable Self-Assem-bly: Constructing Global Shape Using Bio-logically-Inspired Local Interactions andOrigami Mathematics. Ph.D. Dissertation,Massachusetts Institute of Technology,Department of Electrical Engineering andComputer Science. Cambridge, MA.

Pfenning, F., and Schurmann, C. 1999. Sys-tem Description: Twelf—A Beta-LogicalFramework for Deductive Systems. In Pro-ceedings of the 16th International Conferenceon Automated Deduction (CADE-16), 202–206. Albany, NY: International Confer-ences on Automated Deduction, Inc.

Reid, J. R.; Vasilyev, V. S.; and Webster, R.T. 2008. Building Micro-Robots: A Path toSub-mm3 Autonomous Systems. In Techni-cal Proceedings of the 2008 NanotechnologyConference and Trade Show, Volume 1-3.Cambridge, MA: Nano Science and Tech-nology Institute.

Stoy, K., and Nagpal, R. 2004. Self-Recon-figuration Using Directed Growth. In Pro-ceedings of the 7th International Symposiumon Distributed Autonomous Robotic Systems.Berlin: Springer.

Vasilyev, V. S.; Reid, J. R.; and Webster, R.T. 2008. Microfabrication of Si/SIo2-Spher-ical Shells as a Path to Sub-mm3Autonomous Robotic Systems. Paper pre-sented at the Materials Research SocietyFall Meeting, 1–5 December, Boston, MA.

Yim, M.; Shen, W.-M.; Salemi, B.; Rus, D.;Moll, M.; Lipson, H.; Klavins, E.; andChirikjian, G. 2007. Modular Self-Reconfig-urable Robot Systems (Grand Challenges ofRobotics). IEEE Robotics and AutomationMagazine 14(1): 43–52.

Zykov, V., and Lipson, H. 2006. Fluidic Sto-chastic Modular Robotics: Revisiting theSystem Design. Paper presented at the

Articles

44 AI MAGAZINE

and Ph.D in computer science from theUniversity of Michigan.

Rob Reid is a research scientist with the AirForce Research Laboratory, Sensors Direc-torate at Hanscom Air Force Base, MA,where he develops and leads programsapplying microelectromechanical systemstechnology to the development of recon-figurable radio systems. Reid’s current pro-grams include building fully autonomousrobots at the subcubic millimeter scale,developing high-performance, high-powertunable filters, and implementing compo-nents for highly integrated millimeter-wave front ends. In 2007, he received theSensors Directorate Samuel L. Burka awardfor his in-house research efforts.

Daniel D. Stancil is a professor of electricaland computer engineering at CarnegieMellon University. He received his Ph.Dfrom the Massachusetts Institute of Tech-nology in 1981. He has served as associatedepartment head and as associate dean foracademic affairs in the College of Engi-neering. Stancil is an IEEE Fellow and a pastpresident of the IEEE Magnetics Society. Hisresearch interests include wireless commu-nications, antennas, and applied optics.

Michael Philetus Weller is a Ph.D. candi-date in the Computational Design Labora-tory at the Carnegie Mellon School ofArchitecture. His background in architec-ture and the philosophy of cognition hasled to an interest in the ways that thedesigned objects that comprise our physi-cal environment shape our society. Hisresearch currently focuses on using rapidprototyping, robotics and embedded com-putation to create artifacts that supportnovel modes of interaction.

Robotics, Science, and Systems Workshopon Self-Reconfigurable Modular Robots, 19August, Philadelphia, PA.

Seth Copen Goldstein is a professor ofcomputer science at Carnegie Mellon Uni-versity, Pittsburgh, PA. His research inter-ests include computer architecture, compil-ers, reconfigurable computing,programmable matter, and systems nan-otechnology. He received his BSE degree in1985 from Princeton University and hisPh.D. degree from the University of Cali-fornia at Berkeley in 1997.

Todd C. Mowry is a professor in the Com-puter Science Department at Carnegie Mel-lon University. After receiving his Ph.D.from Stanford University in 1994, Mowrywas on the faculty at the University ofToronto before joining Carnegie Mellon in1997. He also served as the director of theIntel Research Pittsburgh lab from 2004through 2007. Mowry’s research interestsspan the areas of computer systems, data-base performance, and modular robotics.He is a member of the editorial board forACM Transactions of Computer Systems.

Jason Campbell is a research scientist atIntel Research Pittsburgh, where hisresearch interests include self-reconfig-urable modular robotics, autonomous sys-tems, embedded computing, and computervision. He previously served as a seniorexecutive in several startup companies,including chief technical officer of ZackSystems and vice president, technology atStrategic Economic Decisions. He holds adegree in electrical engineering from Stan-ford University.

Michael Ashley-Rollman is a graduate stu-dent in the Computer Science Departmentof Carnegie Mellon University. His researchinterests lie in the area of programminglanguages. Currently he is focusing onapproaches for programming massivelydistributed systems, such as those envi-sioned by the Claytronics Project.

Michael De Rosa is a Ph.D. candidate atCarnegie Mellon University, and a memberof the claytronics research group. Hisresearch interests include motion planningfor modular robots, distributed debugging,and high-level languages. He is currentlyinvolved in developing LDP, a declarativelanguage for programming and reasoningin distributed systems.

Stanislav Funiak is a graduate student inthe Robotics Institute at Carnegie MellonUniversity. His research interests lie in algo-rithms for distributed probabilistic infer-ence and machine learning. He is currentlyfocusing on localization in large-scale dis-tributed systems, such as modular robots

and camera networks, and on designingrobust approaches for distributed collabo-rative filtering.

James F. Hoburg is a professor of electricaland computer engineering at CarnegieMellon University. His research interestsare in a wide range of applications of engi-neering electromagnetics, including elec-tric and magnetic field analysis and model-ing, magnetic shielding, discrete andcontinuum electromechanical systems,applied electrostatics, electrohydrodynam-ics and microfluidics. He received a BSEEdegree in 1969 from Drexel University, anSM and EE in 1971, and a Ph.D. in 1975from the Massachusetts Institute of Tech-nology. Hoburg received the Ladd Awardfor excellence in research in 1979 and theRyan Award for excellence in undergradu-ate teaching in 1980, both at Carnegie-Mel-lon. He is an elected Fellow of IEEE.

Mustafa Emre Karagozler is a Ph.D stu-dent in the Department of Electrical andComputer Engineering at Carnegie MellonUniversity, Pittsburgh, PA. His researchinterests are microelectromechanical sys-tems, microrobotics, actuator and sensordesign for microsystems. He holds a BSdegree in electrical and electronics engi-neering from Middle East Technical Uni-versity, Ankara, Turkey, and an MS in elec-trical and computer engineering fromCarnegie Mellon University.

Brian Kirby is a research associate atCarnegie Mellon University specializing inhardware system and printed circuit boarddesign. He received his BS in electrical andcomputer engineering from Carnegie Mel-lon University.

Peter Lee is a professor and head of theComputer Science Department at CarnegieMellon University. His research contribu-tions lie in areas related to the foundationsof software reliability, program analysis,security, and language design. A Fellow ofthe Association for Computing Machineryand chair of the board of directors of theComputing Research Association, Peter Leeis called upon in diverse venues as a con-tributor in research, education, and policymaking.

Padmanabhan (Babu) Pillai has been aresearch scientist at Intel Research Pitts-burgh since 2003, where he has been work-ing on a variety of distributed systems proj-ects. His research interests span a variety ofsystems-related fields, including parallelcomputing, low-power and power-awaresystems, operating systems, and embedded,real-time systems. Babu received a BS inelectrical and computer engineering fromCarnegie Mellon University, and an MS

Articles

SUMMER 2009 45

Support AAAI Open Access

We’re counting on you to helpus deliver the latest informationabout artificial intelligence tothe scientific community, free ofcharge. To enable us to contin-ue this effort, please consider agenerous gift to AAAI. For infor-mation on how you can con-tribute to the open access initia-tive, please see www.aaai.organd click on “Gifts.”