Advances in Dataﬂow Programming Languages - … · Advances in Dataﬂow Programming Languages...

Advances in Dataflow Programming Languages

WESLEY M. JOHNSTON, J. R. PAUL HANNA, AND RICHARD J. MILLAR

University of Ulster

Abstract. Many developments have taken place within dataflow programminglanguages in the past decade. In particular, there has been a great deal of activity andadvancement in the field of dataflow visual programming languages. The motivation forthis article is to review the content of these recent developments and how they cameabout. It is supported by an initial review of dataflow programming in the 1970s and1980s that led to current topics of research. It then discusses how dataflowprogramming evolved toward a hybrid von Neumann dataflow formulation, andadopted a more coarse-grained approach. Recent trends toward dataflow visualprogramming languages are then discussed with reference to key graphical dataflowlanguages and their development environments. Finally, the article details four keyopen topics in dataflow programming languages.

Categories and Subject Descriptors: A.1 [Introductory and Survey]; C.1 [ProcessorArchitectures]; D.2 [Software Engineering]; D.3 [Programming Languages]

General Terms: Languages, Theory

Additional Key Words and Phrases: Dataflow, software engineering, graphicalprogramming, component software, multithreading, co-ordination languages, data flowvisual programming

1. INTRODUCTION

The original motivation for research intodataflow was the exploitation of mas-sive parallelism. Therefore, much workwas done to develop ways to programparallel processors. However, one schoolof thought held that conventional “vonNeumann” processors were inherently un-suitable for the exploitation of parallelism[Dennis and Misunas 1975; Weng 1975].The two major criticisms that were lev-eled at von Neumann hardware were di-rected at its global program counter andglobal updatable memory [Silc et al. 1998],

Authors’ addresses: Faculty of Engineering, University of Ulster, Newtownabbey, Northern Ireland, BT370QB; email: W. M. Johnston, [email protected]; J. R. P. Hanna and R. J. Millar, {p.hanna,rj.millar}@ulster.ac.uk.Permission to make digital/hard copy of part or all of this work for personal or classroom use is grantedwithout fee provided that the copies are not made or distributed for profit or commercial advantage, thecopyright notice, the title of the publication, and its date appear, and notice is given that copying is bypermission of ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists requiresprior specific permission and/or a fee.c©2004 ACM 0360-0300/04/0300-0001 $5.00

both of which had become bottlenecks[Ackerman 1982; Backus 1978]. The al-ternative proposal was the dataflow archi-tecture [Davis 1978; Dennis and Misunas1975; Weng 1975], which avoids both ofthese bottlenecks by using only local mem-ory and by executing instructions as soonas their operands become available. Thename dataflow comes from the conceptualnotion that a program in a dataflow com-puter is a directed graph and that dataflows between instructions, along its arcs[Arvind and Culler 1986; Davis and Keller1982; Dennis 1974; Dennis and Misunas1975]. Dataflow hardware architectures

ACM Computing Surveys, Vol. 36, No. 1, March 2004, pp. 1–34.

2 Johnston et al.

looked promising [Arvind and Culler 1986;Dennis 1980; Treleaven and Lima 1984;Veen 1986], and a number of physical im-plementations were constructed and stud-ied (for examples, see Davis [1978], Keller[1985], Papadopoulos [1988], Sakai et al.[1989], and Treleaven et al. [1982]).

Faced with hardware advances, re-searchers found problems in compil-ing conventional imperative programminglanguages to run on dataflow hardware,particularly those associated with side ef-fects and locality [Ackerman 1982; Arvindet al. 1977; Arvind and Culler 1986;Kosinski 1973; Wail and Abramson 1995;Weng 1975; Whiting and Pascoe 1994].They found that by restricting certainaspects of these languages, such as as-signments, they could create languages[Ackerman 1982; Ashcroft and Wadge1977; Dennis 1974; Hankin and Glaser1981; Kosinski 1978] that more naturallyfitted the dataflow architecture and couldthus run much more efficiently on it. Theseare the so-called dataflow programminglanguages [Ackerman 1982; Whiting andPascoe 1994] that developed distinct prop-erties and programming styles as a conse-quence of the fact that they were compiledinto dataflow graphs—the “machine lan-guage” of dataflow computers.

The often-expressed view in the 1970sand early 1980s that this form of dataflowarchitecture would take over from vonNeumann concepts [Arvind et al. 1977;Treleaven et al. 1982; Treleaven and Lima1984] never materialized [Veen 1986]. Itwas realized that the parallelism usedin dataflow architectures operated at toofine a grain and that better performancecould be obtained through hybrid vonNeumann dataflow architectures. Manyof these architectures [Bic 1990] tookadvantage of more coarse-grained paral-lelism where a number of dataflow in-structions were grouped and executed insequence. These sets of instructions are,nevertheless, executed under the rules ofthe dataflow execution model and thusretain all the benefits of that approach.Most dataflow architecture efforts beingpursued today are a form of hybrid

[Iannucci 1988; Nikhil and Arvind 1989],although not all, for example, Verdosciaand Vaccaro [1998].

The 1990s saw a growth in the field ofdataflow visual programming languages(DFVPLs) [Auguston and Delgado 1997;Baroth and Hartsough 1995; Bernini andMosconi 1994; Ghittori et al. 1998; Greenand Petre 1996; Harvey and Morris 1993,1996; Hils 1992; Iwata and Terada 1995;Morrison 1994; Mosconi and Porta 2000;Serot et al. 1995; Shizuki et al. 2000; Shurr1997; Whiting and Pascoe 1994; Whitley1997]. Some of these, such as LabViewand Prograph were primarily driven byindustry, and the former has become asuccessful commercial product that is stillused today. Other languages, such as NL[Harvey and Morris 1996], were createdfor research. All have software engineer-ing as their primary motivation, whereasdataflow programming was traditionallyconcerned with the exploitation of paral-lelism. The latter remains an importantconsideration, but many DFVPLs are nolonger primarily concerned with it. Ex-perience has shown that many key ad-vantages of DFVPLs lie with the soft-ware development lifecycle [Baroth andHartsough 1995].

This article traces the development ofdataflow programming through to thepresent. It begins with a discussion ofthe dataflow execution model, includinga brief overview of dataflow hardware.Insofar as this research led to the de-velopment of dataflow programming lan-guages, a brief historical analysis of theseis presented. The features that define tra-ditional, textual dataflow languages arediscussed, along with examples of lan-guages in this category. The more recenttrend toward large-grained dataflow ispresented next. Developments in the fieldof dataflow programming languages in the1990s are then discussed, with an empha-sis on DFVPLs. As the environment is keyto the success of a DFVPL, a discussionof the issues involved in development en-vironments is also presented, after whichfour examples of open issues in dataflowprogramming are presented.

ACM Computing Surveys, Vol. 36, No. 1, March 2004.

Advances in Dataflow Programming Languages 3

Fig. 1. A simple program (a) and its dataflow equivalent (b).

2. THE DATAFLOW EXECUTION MODEL

2.1. The Pure Dataflow Model

In the dataflow execution model, a pro-gram is represented by a directed graph[Arvind and Culler 1986; Davis and Keller1982; Dennis 1974; Dennis and Misunas1975; Karp and Miller 1966]. The nodesof the graph are primitive instructionssuch as arithmetic or comparison oper-ations. Directed arcs between the nodesrepresent the data dependencies betweenthe instructions [Kosinski 1973]. Concep-tually, data flows as tokens along thearcs [Dennis 1974] which behave like un-bounded first-in, first-out (FIFO) queues[Kahn 1974]. Arcs that flow toward a nodeare said to be input arcs to that node, whilethose that flow away are said to be outputarcs from that node.

When the program begins, special acti-vation nodes place data onto certain keyinput arcs, triggering the rest of the pro-gram. Whenever a specific set of input arcsof a node (called a firing set) has data on it,the node is said to be fireable [Arvind andCuller 1986; Comte et al. 1978; Davis andKeller 1982]. A fireable node is executed atsome undefined time after it becomes fire-able. The result is that it removes a datatoken from each node in the firing set, per-forms its operation, and places a new data

token on some or all of its output arcs. Itthen ceases execution and waits to becomefireable again. By this method, instruc-tions are scheduled for execution as soonas their operands become available. Thisstands in contrast to the von Neumann ex-ecution model, in which an instruction isonly executed when the program counterreaches it, regardless of whether or not itcan be executed earlier than this.

The key advantage is that, in dataflow,more than one instruction can be executedat once. Thus, if several instructions be-come fireable at the same time, they can beexecuted in parallel. This simple principleprovides the potential for massive parallelexecution at the instruction level.

An example of dataflow versus a tra-ditional sequential program is shown inFigure 1. Figure 1(a) shows a fragment ofprogram code and Figure 1(b) shows howthis is represented as a dataflow graph.The arrows represent arcs, and the circlesrepresent instruction nodes. The squarerepresents a constant value, hard-codedinto the program. The letters representwhere data flows in or out of the rest ofthe program, which is not shown. Wheremore than one arrow emanates from agiven input, it means that the single valueis duplicated and transmitted down eachpath.


4 Johnston et al.

Fig. 2. Gates in a dataflow graph.

Under the von Neumann executionmodel, the program in Figure 1(a) wouldexecute sequentially in three time units.In time unit 1, X and Y are added andassigned to A. In time unit 2, Y is di-vided by 10 and assigned to B. In time unit3, A and B are multiplied together andassigned to C.

Under the dataflow execution model,where the graph in Figure 1(b) is the ma-chine code, the addition and division areboth immediately fireable, as all of theirdata is initially present. In time unit 1, Xand Y are added in parallel with Y be-ing divided by 10. The results are placedon the output arcs, representing variablesA and B. In time unit 2, the multiplica-tion node becomes fireable and is executed,placing the result on the arc representingthe variable C. (In dataflow, every arc canbe said to represent a variable.) In this sce-nario, execution takes only two time unitsunder a parallel execution model.

It is clear that dataflow provides thepotential to provide a substantial speedimprovement by utilizing data dependen-cies to locate parallelism. In addition, ifthe computation is to be performed onmore than one set of data, the calculationson the second wave of values of X andY can be commenced before those on thefirst set have been completed. This isknown as pipelined dataflow [Gao andParaskevas 1989; Wadge and Ashcroft1985] and can utilize a substantial degreeof parallelism, particularly in loops, al-though techniques exist to utilize greaterparallelism in loops [Arvind and Nikhil1990]. A dataflow graph that producesa single set of output tokens for eachsingle set of input tokens is said to bewell-behaved [Dennis 1974; Weng 1975].

Another key point is that the operationof each node is functional. This is because

data is never modified (new data tokensare created whenever a node fires), nonode has any side effects, and the absenceof a global data store means that there islocality of effect. As a result of being func-tional, and the fact that the data travelsin ordered queues, a program expressedin the pure dataflow model is determi-nate [Arvind and Culler 1986; Davis andKeller 1982; Kahn 1974]. This means that,for a given set of inputs, a program willalways produce the same set of outputs.This can be an important property in cer-tain applications. Some research has beendone on the implications of nondetermi-nate behavior [Arvind et al. 1977; Kosinski1978] and this is discussed further inSection 6.4.

2.1.1. Controlling Data Tokens. InFigure 1(b) the two arcs emanatingfrom input Y signify that that value is tobe duplicated. Forking arcs in this man-ner is essential if a data token is neededby two different nodes. A data tokenthat reaches a forked arc gets duplicatedand a copy sent down each branch. Thispreserves the data independence and thefunctionality of the system.

To preserve the determinacy of thetoken-flow model, it is not permitted toarbitrarily merge two arcs of flowing datatokens. If this were allowed, data couldarrive at a node out of order and jeopar-dize the computation. It is obvious, how-ever, that it would be difficult indeed toexpress a program in the dataflow model ifarcs could only be split and never merged.Thus, the dataflow model provides spe-cial control nodes called gates [Davis andKeller 1982; Dennis 1974] that allow thisto happen within well-controlled limits.Figure 2(a) shows a Merge gate. This gates



takes two data input arcs, labeled the trueand false inputs, as well as a “control” in-put arc that carries a Boolean value. Whenthe node fires, the control token is ab-sorbed first. If the value is true, the tokenon the true input is absorbed and placed onthe output. If it is false, then the token onthe false input is absorbed and placedon the output. Figure 2(b) shows a Switchgate. This gate operates in much the sameway, except that there is a single input andthe control token determines on which oftwo outputs it is placed.

A full treatment of controlling tokensto provide conditional and iterative execu-tion is given in Section 6.2. At this stage,it is sufficient to say that by grouping to-gether three Switch gates, it is possibleto implement well-behaved conditional ex-ecution, while the combined use of bothtypes of gate implements well-behaved it-erative execution.

2.1.2. An Alternative to Token-BasedDataflow. The pure dataflow executionmodel outlined above is based on flowingdata tokens, like most dataflow models.However, it should be pointed out thatan alternative, known as the structuremodel, has been proposed in the liter-ature. Expounded by Davis and Keller[1982] and Keller and Yen [1981], itcontains the same arc-and-node formatas the token model. In the structuremodel, however, each node creates onlyone data object on each arc that remainsthere: the node builds one or more datastructures on its output arcs. It is possi-ble for these structures to hold infinitearrays of values, permitting open-endedexecution, and creating the same effect asthe token model, but with the advantagethat the structure model permits randomaccess of the data structures and historysensitivity.

The key difference between the struc-ture model and the token model is the waythey view data. In the token model, nodesare designed to be stream processors, oper-ating on sequences of related data tokens.In the structure model, however, the nodesoperate on structures and have no concept

of streams of data structures. As a conse-quence, a structure model will need a morecomplex supporting language [Davis andKeller 1982].

Initially, the structure model seems at-tractive. Token streams can be repre-sented by infinite objects with the advan-tage that the streams can be accessedrandomly and that the entire history ofa stream can be accessed without need-ing to explicitly preserve earlier data fromthe stream. Additionally, the point is madethat token models force the programmerto model all programs as token streams,while the structure model allows them tomake the choice [Davis and Keller 1982].

However, the structure model has thekey disadvantage that it cannot store dataefficiently. It requires all data generatedto be stored, to preserve the ability to ex-amine the history of the program. Tokenmodels are inherently more efficient atstoring data. Some of this problem canbe alleviated by compiler efficiency, but itis a complex process. Despite research inthe area in the early 1980s [Davis andKeller 1982; Keller and Yen 1981], thestructure model was not widely adoptedinto dataflow, which remains almost exclu-sively token-based.

2.1.3. Theoretical Implementation: Data andDemand-Driven Architectures. The earliestdataflow proposals imagined data tokensto be passive elements that remained onarcs until they were read, rather than ac-tually controlling the execution [Kosinski1973]. However, it quickly became normalin dataflow projects for data to control theexecution. There were two ways of doingthis in a theoretical implementation of thepure dataflow model.

The first approach is known as thedata-driven approach [Davis and Lowder1981; Davis and Keller 1982; Dennis 1974;Treleaven et al. 1982], although this termis slightly misleading as both approachescan be said to be driven by data, in-sofar as they follow the principles ofdataflow. This approach should really betermed the data-availability-driven ap-proach because execution is dependent on


6 Johnston et al.

the availability of data. Essentially a nodeis inactive while data is arriving at its in-puts. It is the responsibility of an over-all management device to notify and firenodes when their data has arrived. Thedata-driven approach is a two-phase pro-cess where (a) a node is activated when itsinputs are available, and (b) absorbs its in-puts and places tokens on its output arcs.

The second approach is the demand-driven approach [Davis and Keller 1982;Kahn 1974]. In this approach, a node isactivated only when it receives a requestfor data from its output arcs. At this point,it demands data from all relevant inputarcs. Once it has received its data, it exe-cutes and places data tokens on its outputarcs. The demand-driven approach is thusa four-phase process [Davis and Keller1982] where (a) a node’s environment re-quests data, (b) the node is activated andrequests data from its environment, (c) theenvironment responds with data, and (d)the node places tokens on its output arcs.Execution of the program begins when thegraph’s environment demands some out-put from the graph.

Each of these approaches has certain ad-vantages. The data-driven approach hasthe advantage that it does not have the ex-tra overhead of propagating data requestsup the dataflow graph. On the other hand,the demand-driven approach has the ad-vantage that certain types of node can beeliminated, as pointed out by Davis andKeller [1982]. This is because only neededdata is ever demanded. For example, theSwitch, node, shown in Figure 2(b), is notrequired under a demand-driven approachbecause only one of the True or False out-puts will demand the input, but not both.Therefore, they can both be attached di-rectly to the input. This is one example ofhow programming with dataflow can be af-fected by the choice of physical implemen-tation, or at least by the choice of executionmodel.

It can also be argued that the demand-driven approach prevents the creation ofcertain types of programs. For example,modern software is often event-driven,such as for business software or real-timesystems. It is not enough for the output

environment to simply demand input.These examples seem to require a data-driven approach.

2.2. Early Dataflow Hardware Architectures

While dataflow seems good in theory,the practical implementation of the puredataflow model has been found to be anarduous task [Silc et al. 1998]. There area number of reasons for this, primarilythe fact that the pure model makes as-sumptions that cannot be replicated in thereal world. First, it assumes that the arcsare FIFO queues of unbounded capacity,but creating an unbounded memory is im-possible in a practical sense. Thus anydataflow implementation is heavily tiedto token-storage techniques. Second, it as-sumes that any number of instructions canbe executed in parallel, while in realitythe number of processing elements willbe finite. These restrictions mean that nohardware implementation of the dataflowmodel will exactly mirror the pure model.Indeed, this fact can make subtle butimportant changes to the pure dataflowmodel that mean that the implementa-tion may deadlock in cases where the puremodel predicts no deadlock [Arvind andCuller 1986]. It is useful to summarize theearly development of dataflow hardwarein order to reinforce this point.

2.2.1. The Static Architecture. When theconstruction of dataflow computers beganin the 1970s, two different approaches tosolving the previously mentioned prob-lems were researched. The static architec-ture was proposed by Dennis and Misunas[1975]. Under this architecture [Dennis1974, 1980; Silc et al. 1998], the FIFO de-sign of arcs is replaced by a simpler de-sign where each arc can hold, at most,one data token. The firing rule for a nodeis, therefore, that a token be present oneach input arc, and that there be no to-kens present on any of the output arcs. Inorder to implement this, acknowledge arcsare implicitly added to the dataflow graphthat go in the opposite direction to eachexisting arc and carry an acknowledgmenttoken.



Fig. 3. The static dataflow architecture (based on Arvind and Culler [1986]).

The static architecture’s main strengthis that it is very simple and quick to de-tect whether or not a node is fireable. Addi-tionally, it means that memory can be allo-cated for each arc at compile-time as eacharc will only ever hold 0 or 1 data token.This implies that there is no need to createcomplex hardware for managing queues ofdata tokens: each arc can be assigned to aparticular piece of memory store.

The graph itself is stored in the com-puter as a series of templates, each repre-senting a node of the graph. The templateholds an opcode for the node; a memoryspace to hold the value of the data tokenon each input arc, with a presence flagfor each one; and a list of destination ad-dresses for the output tokens. Each tem-plate that is fireable (the presence flag foreach input is set, and that of each out-put is not set) has its address placed inan instruction queue. A fetch unit thenrepeatedly removes each template fromthis queue and sends an operation packetto the appropriate operation unit. Mean-while, the template is cleared to prepare itfor the next set of data tokens. The result issent from the operation unit to an updateunit that places the results onto the cor-rect receiving arcs by reading the targetaddresses in the template. It then checkseach template to see if it is fireable and,if so, places it in the instruction queue tocomplete the cycle. This process is shownin Figure 3.

Unfortunately the static model hassome serious problems. The additional

acknowledgment arcs increase data trafficin the system, without benefiting the com-putation. According to Arvind and Culler[1986], traffic can increase by a factor of1.5 to 2.0. Because a node must wait foracknowledgment tokens to arrive beforeit can execute again, the time betweensuccessive firings of a node increases.This can affect performance, particularlyin situations of linear computation thatdo not have much parallelism. Perhapsmost importantly, the static architecturealso severely limits the execution of loops.In certain cases, the single-token-per-arclimitation means that a second loop itera-tion cannot begin executing until the pre-vious one has almost completed, therebylimiting parallelism to simple pipeliningand preventing truly parallel execution ofloop iterations. Despite these limitations,a number of static dataflow computershave been built and studied [Davis 1978;Dennis and Misunas 1975; Dennis 1980].

2.2.2. The Dynamic, or Tagged-TokenArchitecture. An alternative approach wasproposed by Watson and Gurd [1979],Arvind and Culler [1983], and Arvindand Nikhil [1990]. Known as the dynamicmodel, it exposes additional parallelismby allowing multiple invocations of a sub-graph that is often an iterative loop. Whilethis is the conceptual view of the tagged-token model, in reality only one copy ofthe graph is kept in memory and tags areused to distinguish between tokens that


8 Johnston et al.

belong to each invocation. A tag containsa unique subgraph invocation ID, as wellas an iteration ID if the subgraph is aloop. These pieces of information, takentogether, are commonly known as thecolor of the token.

Instead of the single-token-per-arc ruleof the static model, the dynamic model rep-resents each arc as a large bag that cancontain any number of tokens, each witha different tag [Silc et al. 1998]. In thisscenario, a given node is said to be fire-able whenever the same tag is found in adata token on each input arc. It is impor-tant to note that, because the data tokensare not ordered in the tagged-token model,processing of tokens does not necessarilyproceed in the same order as they enteredthe system. However, the tags ensure thatthe tokens do not conflict, so this does notcause a problem.

The tags themselves are generated bythe system [Arvind and Culler 1986]. To-kens being processed in a given invoca-tion of a subgraph are given the uniqueinvocation ID of that subgraph. Their it-eration ID is set to zero. When the tokenreaches the end of the loop and is being fedback into the top of the loop, a special con-trol operator increments the iteration ID.Whenever a token finally leaves the loop,another control operator sets its iterationID back to zero.

A hardware architecture based on thedynamic model is necessarily more com-plex than the static architecture outlinedin Section 2.2.1. Additional units are re-quired to form tokens and match tags.More memory is also required to store theextra tokens that will build up on the arcs.Arvind and Culler [1986] provided a goodsummary of the architecture.

The key advantage of the tagged-tokenmodel is that it can take full advantageof pipelining effects and can even executeseparate loop iterations simultaneously. Itcan also execute out-of-order, bypassingany tokens that require complex executionand that delay the rest of the computa-tion. It has been shown that this modeloffers the maximum possible parallelismin any dataflow interpreter [Arvind andGostelow 1977].

Another noteworthy benefit of thetagged-token model is that less care needsto be taken to ensure that tokens re-main in order. For example, the puredataflow model requires Merge operators(see Section 2.1.1) to ensure that data to-kens are merged in a determinate way. Inthe dynamic model, however, this is notrequired as the tags ensure the determi-nacy, and so token streams can be mergedarbitrarily.

The main disadvantage of the tagged-token model is the extra overhead re-quired to match tags on tokens, insteadof simply their presence or absence. Morememory is also required and, due to thequantity of data being stored, an asso-ciative memory is not practical. Thus,memory access is not as fast as it couldbe [Silc et al. 1998]. Nevertheless, thetagged-token model does seem to of-fer advantages over the static model. Anumber of computers using this modelhave been built and studied [Arvindand Culler 1983; Barahona and Gurd1985].

As stated above, the choice of target ar-chitecture can have implications on theprogramming of software. Depending onthe model chosen, certain types of nodes,such as merge or switch nodes, are not re-quired. Additionally, the performance ofthe program will be affected and someproperties of the system (such as its ten-dency to deadlock, which can be veri-fied under the pure dataflow model) maychange subtly under certain implemen-tations [Arvind and Culler 1986; Naggaret al. 1999]. For example, some networksthat deadlock under the static model maynot deadlock under the dynamic model.This is due to the pure model’s theoret-ically valid but impractical assumptionsthat there are an infinite number of pro-cessing elements and infinite space oneach arc [Kahn 1974].

2.3. Synchronous Dataflow

A later development in dataflow, butone that became quite widely used, wassynchronous dataflow (SDF) [Lee andMesserschmitt 1987]. This is a subset of



the pure dataflow model in which thenumber of tokens consumed and producedon each arc of a node is known at compile-time [Bhattacharyya 1996]. Under SDF,the number of tokens initially on each arcis also specified at compile-time. In thisscenario, there are certain limitations thatmean that some kinds of program can-not be represented. For example, loops canonly be specified when the number of iter-ations is known at compile-time.

The advantage of the SDF approach,however, is that it can be statically sched-uled [Buck and Lee 1995]. This meansthat it can be converted into a sequentialprogram and does not require dynamicscheduling. It has found particular appli-cations in digital signal processing wheretime is an important element of the com-putation [Lee and Messerschmitt 1987;Plaice 1991]. Even dataflow graphs whichare not SDF in themselves may havesubgraphs that are, and this may allowpartial static scheduling [Buck and Lee1995], with the rest scheduled according tothe usual dataflow scheduling techniques.This has applications to coarse-graineddataflow discussed in Section 4.3.

3. EARLY DATAFLOW PROGRAMMINGLANGUAGES

3.1. The Development of DataflowLanguages

With the development of dataflow hard-ware came the equally challenging prob-lem of how to program these machines.Because they were scheduled by data de-pendencies, it was clear that the program-ming language must expose these depen-dencies. However, the data dependenciesin each class of language can be exploitedto different degrees, and the amount ofparallelism that can be implicitly or ex-plicitly specified also differs. Therefore,the search began for a suitable paradigmto program dataflow computers and asuitable compiler to generate the graphs[Arvind et al. 1988]. Various paradigmswere tried, including imperative, logical,and functional methods. Eventually, themajority consensus settled on a specific

type of functional language that becameknown as dataflow languages.

An important clarification must bemade at this stage. In early publica-tions, dataflow graphs are often used toillustrate programs. In many cases, thesegraphs are simply representations of thecompiled code [Dennis and Misunas 1975]that would be executed on the machine,where the graph was generated eitherby hand or by a compiler from a third-generation programming language. Un-til the advent of Dataflow Visual Pro-gramming Languages in the 1980s and1990s, it was rarely the intention of re-searchers that developers should gener-ate these graphs directly. Therefore theseearly graphs are not to be thought of as“dataflow programming languages.”

3.1.1. What Constitutes a Dataflow Program-ming Language?. While dataflow programscan be expressed graphically, most of thelanguages designed to operate on dataflowmachines were not graphical. There aretwo reasons for this. First, at the low levelof detail that early dataflow machines re-quired, it became tedious to graphicallyspecify constructs such as loops and datastructures which could be expressed moresimply in textual languages [Whiting andPascoe 1994]. Second, and perhaps moreimportantly, the hardware for displayinggraphics was not available until relativelyrecently, stifling any attempts to developgraphical dataflow systems. Therefore,traditional dataflow languages are pri-marily text-based.

One of the problems in defining ex-actly what constitutes a dataflow lan-guage is that there is an overlap with otherclasses of language. For example, the useof dataflow programming languages is notlimited to dataflow machines. In the sameway, some languages, not designed specifi-cally for dataflow, have subsequently beenfound to be quite effective for this use (e.g.,Ashcroft and Wadge [1977]; Wadge andAshcroft [1985]). Therefore, the bound-ary for what constitutes a dataflow lan-guage is somewhat blurred. Nevertheless,there are some core features that would


10 Johnston et al.

appear to be essential to any dataflow lan-guage. The best list of features that con-stitute a dataflow language was put for-ward by Ackerman [1982] and reiteratedby Whiting and Pascoe [1994] and Wailand Abramson [1995]. This list includesthe following:

(1) freedom from side effects,(2) locality of effect,(3) data dependencies equivalent to sche-

duling,(4) single assignment of variables,(5) an unusual notation for iterations due

to features 1 and 4,(6) lack of history sensitivity in proce-

dures.

Because scheduling is determined fromdata dependencies, it is important thatthe value of variables do not change be-tween their definition and their use. Theonly way to guarantee this is to disallowthe reassignment of variables once theirvalue has been assigned. Therefore, vari-ables in dataflow languages almost uni-versally obey the single-assignment rule.This means that they can be regarded asvalues, rather than variables, which givesthem a strong flavor of functional pro-gramming. The implication of the single-assignment rule is that the compiler canrepresent each value as one or more arcs inthe resultant dataflow graph, going fromthe instruction that assigns the value toeach instruction that uses that value.

An important consequence of the single-assignment rule is that the order of state-ments in a dataflow language is notimportant. Provided there are no circularreferences, the definitions of each value,or variable, can be placed in any order inthe program. The order of statements be-comes important only when a loop is be-ing defined. In dataflow languages, loopsare usually provided with an imperativesyntax, but the single-assignment rule ispreserved by using a keyword such as nextto define the value of the variable on thenext iteration [Ashcroft and Wadge 1977].A few dataflow languages offer recursioninstead of loops [Weng 1975].

Freedom from side effects is also es-sential if data dependencies are to de-termine scheduling. Most languages thatavoid side effects do so by disallowingglobal variables and introducing scoperules. However, in order to ensure the va-lidity of data dependencies, a dataflow pro-gram does not even permit a function tomodify its own parameters. All of this canbe avoided by the single-assignment rule.However, problems arise with this strat-egy when data structures are being dealtwith. For example, how can an array bemanipulated if only one assignment canever be made to it? Theoretically, this prob-lem is dealt with by conceptually viewingeach modification of an array as the cre-ation of a new copy of the array, with thegiven element modified. This issue is dealtwith in more detail in Section 6.3.

It is clear from the above discussion thatdataflow languages are almost invariablyfunctional. They have applicative seman-tics, are free from side effects, are determi-nate in most cases, and lack history sen-sitivity. This does not mean that dataflowand functional languages are equivalent.It is possible to write certain convo-luted programs in the functional languageLucid [Ashcroft and Wadge 1977], whichcannot be implemented as a dataflowgraph [Ashcroft and Wadge 1980]. At thesame time, much of the syntax of dataflowlanguages, such as loops, has been bor-rowed from imperative languages. Thus itseems that dataflow languages are essen-tially functional languages with an imper-ative syntax [Wail and Abramson 1995].

3.1.2. Dataflow Languages. A number oftextual dataflow languages, or functionallanguages that can be used with dataflow,have been implemented. A representativesample is discussed below. (Whiting andPascoe [1994] presented a fuller reviewof these languages.) Dataflow Visual Pro-gramming Languages are discussed in de-tail in Section 5.

—TDFL. The Textual Data-Flow Lan-guage was developed by Weng [1975] asone of the first purpose-built dataflowlanguages. It was designed to be



compiled into a dataflow graph withdata streams in a relatively straight-forward way and supported compile-time deadlock detection. A programexpressed in TDFL consisted of a se-ries of modules, analogous to proce-dures in other languages. Each modulewas made up of a series of statementsthat were either assignments (obeyingthe single-assignment rule), conditionalstatements, or a call to another mod-ule. Iteration was not provided directly,as Weng could find no way to make itcompatible with the single-assignmentrule, but modules could call themselvesrecursively.

—LAU. Developed in 1976 for theLAU static dataflow architecture,the LAU language was developed bythe Computer Structures Group ofONERA-CERT in France [Comte et al.1978; Gelly 1976]. It was a single-assignment language and includedconditional branching and loops thatwere compatible with this rule throughthe use of the old keyword. It was oneof the few dataflow languages thatprovided explicit parallelism throughthe expand keyword that specifiedparallel assignment. LAU had somefeatures that were similar to object-oriented languages, such as the abilityto encapsulate data and operations[Comte et al. 1978].

—Lucid. Originally developed indepen-dently of the dataflow field by Ashcroftand Wadge [1977], Lucid was a func-tional language designed to enable for-mal proofs. Recursion was regarded astoo restrictive for loop constructs, butit was realized that iteration introducedtwo nonmathematical features into pro-gramming: transfer and assignment.Thus, Lucid was designed to permit iter-ation in a way that was mathematicallyrespectable, through single assignmentand the use of the keyword next to definethe value of the variable in the next iter-ation. It quickly became apparent, how-ever, that Lucid’s functional and single-assignment semantics were similar tothose required for dataflow machines,

and Ashcroft and Wadge [1980] broodedon the topic in literature before publish-ing a book in 1985 [Wadge and Ashcroft1985] that firmly established Lucid’sclaim to be a dataflow language.

—Id. Originally developed by Arvind et al.[1978] for writing operating systems, Idwas intended to be a language with-out either sequential control or memorycells, two aspects of the von Neumannmodel that Arvind et al. felt must berejected. The resultant language hadsingle-assignment semantics and wasblock-structured and expression-based.Id underwent much evolution, and laterversions tackled the problem that datastructures were not comfortably com-patible with the single-assignment rulethrough the inclusion of I-structures[Arvind et al. 1989] (which are them-selves functional data structures andare explained in Section 6.3).

—LAPSE. Developed by Glauert [1978],LAPSE was derived from Pascal andwas designed for use on the Manchesterdataflow machine. The language hadsingle-assignment semantics and pro-vided functions, conditional evaluation,and user-defined data types. It providediteration without using any qualifyingkeywords to differentiate between thecurrent and next value of the loop vari-able. Rather, the compiler assumed thatthe old value was intended if it appearedin an expression, and the next value wasassumed if it appeared on the left of anassignment. Like LAU, LAPSE provideda single explicit parallel construct, forall for parallel array assignment.

—VAL. VAL was developed by Dennisstarting in 1979 [Ackerman andDennis 1979; Dennis 1977], and obeyedthe single-assignment rule. A pro-gram in VAL consisted of a series offunctions, each of which could returnmultiple values. Loops were providedby the Lucid technique [Ashcroft andWadge 1977], and a parallel assignmentconstruct, for all, was also provided.However, recursion was not providedas it was not thought necessary for thetarget domain. Other disadvantages


12 Johnston et al.

[Whiting and Pascoe 1994] included thelack of general I/O and the fact thatnondeterministic programs could notbe expressed.

—Cajole. First developed under this namein 1981 [Hankin and Glaser 1981],Cajole was a functional language de-signed to be compiled into acyclicdataflow graphs. It did not provideloops, but did permit recursion. Cajolewas later used in a project that exploredstructured programming with dataflow[de Jong and Hankin 1982].

—DL1. Developed by Richardson [1981] tosupport research into hybrid dataflowarchitectures, DL1 was a functional lan-guage designed to be compiled intolow-level dataflow graphs. This tar-get was made more explicit than inother languages, as evidenced by key-words such as subgraph. The languageprovided for recursion and conditionalexecution.

—SISAL. Like Lucid, SISAL was not orig-inally written specifically for dataflowmachines, but found that applicationlater. Originally developed in 1983[Gurd and Bohm 1987; McGraw et al.1983], SISAL is a structured functionallanguage, providing conditional evalu-ation and iteration consistent with thesingle-assignment rule. Although it pro-vides data structures, these are treatedas values and thus cannot be rewrit-ten like I-structures [Arvind et al. 1989].The only parallel construct provided isa parallel loop.

—Valid. Designed by Amamiya et al.[1984], Valid was an entirely functionallanguage designed to demonstrate the“superiority” of dataflow machines. Itprovided recursion as a key languageelement, but also provided functionalloops using the Lucid method [Ashcroftand Wadge 1977]. A simple parallel loopconstruct was also provided.

The above list represents much of thepopulation of dataflow programming lan-guages in existence. Many are similar andthe majority have (1) functional seman-tics, (2) single assignment of variables,

and (3) limited constructs to supportconcurrency.

3.1.3. Using Imperative Languages withDataflow. While the majority consensussettled on the previously mentioneddataflow languages, this does not meanthat other directions were not pursued.Dataflow compilers have been built forseveral imperative languages [Wail andAbramson 1995]. These include For-tran, Pascal, and several dialects of C[Whiting and Pascoe 1994]. All of theseapproaches had to deal with the majorproblem of how to generate code based ondata dependencies from languages thatallow a lot of flexibility in this regard.Wail and Abramson [1995] confirmedthat when programming dataflow ma-chines with imperative languages, thegeneration of good parallel code can beextremely difficult. They also confirmedthat the implementation of nonfunctionalfacilities, such as global variables, willreduce possible concurrency. Their mainmotivation for pursuing this line of re-search was that much software is alreadywritten in imperative languages and mostprogrammers are already familiar withthe paradigm.

In their 1982 paper, Gajski et al. [1982]offered the opinion that using dataflowlanguages offered few advantages overimperative languages, on the groundsthat the compiler technology was as com-plex for one as for the other. They ar-gued that the use of sophisticated com-piler techniques and explicit concurrencyconstructs in an imperative languagecould provide the same level of paral-lel performance on dataflow machines asa dataflow language. While not deny-ing the advantages of the syntacticalpurity of dataflow languages, they ar-gued that these advantages do not jus-tify the effort required for the introduc-tion of a totally new class of programminglanguages.

The possibility of placing explicit con-currency constructs into dataflow lan-guages has been largely resisted by re-searchers (parallel assignment/loops havebeen provided in some cases—see Dennis



[1977], Gelly [1976], and Glauert [1978]—but almost no other concurrency featureshave been included). It is probable thatthe explanation for this resistance is thatimplicit parallelism is an extremely ap-pealing idea, and to introduce explicitconcurrency constructs into their dataflowlanguages would destroy one of the mostappealing parts of the dataflow concept.

The argument of Gajski et al. [1982]against creating specifically “dataflow”programming languages would be validif the only justification for the pursuit ofdataflow were the pursuit of improved per-formance through exploiting parallelism.However, as they themselves commented,dataflow languages have features that arevery advantageous to the programmer.The future development of dataflow visualprogramming languages provides muchevidence for this, where the emphasis hasmoved toward benefits in software engi-neering. Therefore, contrary to the asser-tions of Gajski et al. [1982] it is proposedthat further research into dataflow pro-gramming languages is justified.

3.2. The Dataflow Experience in the 1980s

In the 1980s, proponents confidentlypredicted that both dataflow hardwareand dataflow languages would supersedevon Neumann-based processors and lan-guages [Arvind et al. 1977; Treleaven et al.1982; Treleaven and Lima 1984]. How-ever, looking back over the 1990s, it is clearthat this did not happen [Silc et al. 1998;Veen 1986; Whiting and Pascoe 1994].In fact, research into dataflow languagesslowed after the mid-1980s. In their re-view paper of 1994, Whiting and Pascoe[1994] reported that several dataflow re-searchers had concluded that dataflowhad been mostly a failure—cost-effectivedataflow hardware had failed to materi-alize. Given that the dataflow conceptslooked promising, the languages were ap-pealing, and much research effort was putinto the subject, there must be good rea-sons for the decline of these early dataflowconcepts.

It is widely believed that a main rea-son was that the early dataflow hardware

architectures operated at a level that wastoo fine-grained. While von Neumann ar-chitectures operate at process-level gran-ularity (i.e., instructions are groupedinto threads or processes and then exe-cuted sequentially), dataflow operates atinstruction-level granularity. This pointhad been recognized by 1986, when Veen[1986, p. 393] remarked that “there aresigns that a deviation is also necessaryfrom the fine-grain approach” becauseit led to “excessive consumption of re-sources.” This is because it required a highlevel of overhead to prepare each instruc-tion for execution, execute it, propagatethe resultant tokens, and test for furtherenabled firings. Indeed, algorithms whichexhibit a low degree of natural paral-lelism can execute unacceptably slowly ondataflow machines because of this degreeof overhead. Veen [1986] defended theseclaims, arguing that the overhead can bereduced to an acceptable level by com-piling techniques, but later experiencesseem to demonstrate that the criticismwas valid. For example, Bic [1990, p. 42]commented that “inefficiencies [are] in-herent to purely dataflow systems” whileSilc et al. [1998, p. 9] commented that“pure dataflow computers . . . usually per-form quite poorly with sequential code.”

The reason for the decline in dataflowresearch in the late 1980s and early1990s was almost entirely due to prob-lems with the hardware aspects of thefield. There was little criticism of dataflowlanguages—other than those leveled atfunctional languages in general [Gajskiet al. 1982]—which are still unrivaled inthe degree of implicit parallelism that theyachieve [Whiting and Pascoe 1994]. Thedataflow execution model can be used withor without dataflow hardware, and, there-fore, any decline in the hardware aspectdoes not necessarily affect dataflow lan-guages, provided they have advantages ontheir own merit. This article contends thatthey do.

4. EVOLUTION OF DATAFLOW

It is not surprising, on the basis ofSection 3.2, that when research into


14 Johnston et al.

dataflow again intensified in the early1990s [Lee and Hurson 1993], the is-sue of granularity was one of the keypoints to be addressed. One of the pri-mary realizations that made this shiftpossible was the recognition that, con-trary to what was popularly believedin the early 1980s, dataflow and vonNeumann techniques were not mutuallyexclusive and irreconcilable concepts, butsimply the two extremes of a contin-uum of possible computer architectures[Papadopoulus and Traub 1991; Silc et al.1998].

Fine-grained dataflow could now beseen as a multithreaded architecture inwhich each machine-level instruction wasexecuted in a thread on its own. At thesame time, von Neumann architecturescould now be regarded as a multithreadedarchitecture in which there was only onethread—the program itself. For example,in their survey paper, Lee and Hurson[1993, p. 286] observed that “the foremostchange is a shift from the exploitationof fine- to medium- and large-grain par-allelism.” The primary issue in dataflowthus immediately became the question ofgranularity.

The result of this shift in viewpoint wasthe exploration of what has become knownas hybrid dataflow.

4.1. The Development of Hybrid Dataflow

Although hybrid dataflow concepts hadbeen explored for many years [Silc et al.1998], it was only in the 1990s that theybecame the dominant area of research inthe dataflow community. In their 1995 pa-per, Sterling et al. [1995] explored the per-formance of different levels of granularityin dataflow machines. Although the rangeof possible test scenarios is huge, they didproduce a generalized graph that summa-rized their findings. A simplified versionof this is shown in Figure 4.

Figure 4 indicates that neither fine-grained (as in traditional dataflow) norcoarse-grained (as in sequential execu-tion) dataflow offers the best parallel per-formance, but rather a medium-grained

Fig. 4. Dataflow granularity optimization curve,(based on Sterling et al. [1995]).

approach should be used. This suggeststhat some form of hybrid—dataflow withvon Neumann extensions, or vice versa—would offer the best performance. Thequestion then was, what level of mediumgranularity was best?

In terms of hardware architectures,there is no universal consensus on howbest to achieve this hybrid. Some ap-proaches are essentially von Neumannarchitectures with a few dataflow addi-tions. Others are essentially dataflow ar-chitectures with some von Neumann addi-tions. (For examples see Iannucci [1988];Nikhil and Arvind [1989]; Papadopoulosand Traub [1991]). It is not our intentionto explore the hardware aspects of hybriddataflow in depth here, as the concentra-tion of this article is on dataflow program-ming, but a good summary was publishedby Silc et al. [1998].

While this new research was aimedat improving hardware architectures, therejection of fine-grained dataflow andthe move toward more coarse-grainedexecution has also freed the dataflow pro-gramming of its restriction to fine-grainedexecution. This allows a much wider rangeof research to be conducted into dataflowprogramming, taking advantage of thesenew degrees of granularity.

Essentially the now-accepted require-ment for more coarse-grained execu-tion has caused a divergence in thedataflow programming community. Thefirst group advocates generating fine-grained dataflow graphs as before, but



they then propose analyzing these graphsand identifying subgraphs that exhibitlow levels of parallelism that should al-ways execute in sequence. These nodesare grouped together into segments. Thus,when the first node in the segment isfired, the remaining nodes can be firedimmediately. They are still executed ina fine-grained manner, but the costlytoken-matching process is avoided for thesubsequent nodes in the sequence, sav-ing time and resources. This approachis termed threaded dataflow [Silc et al.1998].

The second group advocates dispens-ing with fine-grained dataflow execution,and instead compiling the subgraphs intosequential processes. These then becomecoarse-grained nodes, or macroactors. Thegraphs are executed using the traditionaldataflow rules, with the only differencebeing that each node contains, for exam-ple, an entire function expressed in a se-quential language as opposed to a sin-gle machine-level instruction. This secondapproach is usually termed large-graindataflow [Silc et al. 1998].

It was noted as early as 1974 that themathematical properties of dataflow net-works are valid, regardless of the degreeof granularity of the nodes [Arvind et al.1988; Jagannathan 1995; Kahn 1974; Lee1997; Sterling et al. 1995], and, therefore,the hybrid approaches to dataflow pro-gramming do not in any way compromisethe execution model. Both the threadedand large-grain approaches are excitingdevelopments but it is the latter that of-fers the most potential for improvementsto dataflow programming.

4.2. Threaded Dataflow

The threaded dataflow approach takes ad-vantage of the fact that dataflow programgraphs display some level of sequential ex-ecution. For example, in the case wherethe output of one node goes into the nextnode, the two nodes could never executein parallel when they are operating ona single wave of data. Therefore, there

is little point in scheduling them to twodifferent processors. Under fine-graineddataflow, the output token from the firstnode must be mapped back through thesystem, added to the input arc for the sec-ond node, which must then wait to be fired.It is much more efficient to place the twoinstructions in a single execution quanta,so that the output of the first node canbe immediately used by the second [Bic1990].

This principle was used by Bic [1990],who proposed analyzing a dataflow graphand producing sequential code segments(SCS), which are nodes that are in a chainand cannot be executed in parallel. Underthe modified execution model, the granu-larity is at the SCS level. However, otherthan the fact that the execution of the firstinstruction in an SCS causes the rest ofthe chain to be executed, the model obeysthe standard dataflow rules. Bic [1990]also proposed a method to automaticallyidentify the SCSs without programmerintervention.

The advantage of the threaded dataflowapproach is that those parts of thedataflow graph that do not exhibit good po-tential parallelism can be executed with-out the associated overhead, while thosethat do show potential parallelism cantake advantage of it. In their study ofthis approach, Papadopoulus and Traub[1991] confirmed these conclusions, al-though they warned that it is not wiseto carry the line of sequentiality toofar. An analysis of these proposals un-dertaken by Bohm et al. [1993] demon-strated that medium-grained dataflow didindeed improve performance, as predictedby Sterling et al. [1995].

A key open question is how best to par-tition programs into threads and whatdegree of granularity is best [Bohmet al. 1993; Lee and Hurson 1994]. ThePebbles group in the U.S. is examin-ing the relationship between granular-ity of parallelism and efficiency in hybriddataflow [Bohm et al. 1993; Najjar et al.1994], although the group is primarily con-cerned with large-grain dataflow. Anotherconsequent benefit of this area of research


16 Johnston et al.

has been that it is easier to encode certainfunctions (e.g., resource management) ifsome sequential execution is permitted[Lee and Hurson 1994].

4.3. Large-Grain Dataflow

Large-grain dataflow can begin witha fine-grained dataflow graph. Thisdataflow graph is analyzed and dividedinto subgraphs, much like the threadedapproach. However, instead of remainingas groupings of associated nodes, thesubgraphs are compiled into sequentialvon Neumann processes. These are thenrun in a multithreaded environment,scheduled according to the usual dataflowprinciples. The processes are termedmacroactors [Lee and Hurson 1994].Much recent work has been done in thearea of large-grain dataflow systemsand it offers a great opportunity forimprovements to the field of dataflowprogramming.

One important point is that, since themacroactors in large-grain dataflow aresections of sequential code, there is no rea-son why these have to be derived from fine-grained dataflow graphs. The macroactorscould just as easily be programmed in animperative language, such as C or Java.Each macroactor could represent an en-tire function, or part of a function, andcould be designed to be used as off-the-shelf components. It is the fact that themacroactors are still executed accordingto dataflow rules that lets this approachretain the clear advantages of dataflow,but solve the high-overhead problemof fine-grained dataflow. Research hasbeen conducted into the best degree ofgranularity by Sterling et al. [1995],who found a medium-grained approachto be optimal.

A related field is the “flow-based pro-gramming” methodology advocated byMorrison [1994]. Morrison advocated theuse of large-grain components, expressedin an imperative language, but linked to-gether in a way reminiscent of dastaflow.Although it is not dataflow—it does notstrictly obey the dataflow firing rules—Morrison’s proposals do suggest features

that could be incorporated into dataflow,including greatly simplified schemes forproviding iteration and the use of sub-streams within streams. The book empha-sizes the benefits of these principles tobusiness software, and to software engi-neering in general.

The key benefit of Morrison’s [1994] ap-proach is a much reduced developmenttime. Empirical evidence of this is of-fered in his book, where real-life experi-ence of a large piece of software devel-oped with flow-based techniques is cited.The approach led to a considerable sav-ings of time and effort on the part ofthe programmers, particularly when itcame to modifying the program after ini-tial completion. Morrison expounded atlength on the benefits of the stream-based,large-grained modular approach to busi-ness software engineering in particular.If these concepts were applied to large-grain dataflow, the advantages of bothdataflow execution and Morrison’s graph-ical component-based software could bemerged.

Similar techniques have alreadybeen used in one key area—digital sig-nal processing [Bhattacharyya 1996;Naggar et al. 1999]. Many signal-processing environments, such as Ptolemy[Bhattacharyya 1996], operate by lettingthe user connect together components,each of which performs a medium-grainedprogramming task. The whole networkis essentially a dataflow network. Somework has been done in formalizing suchnetworks [Lee and Parks 1995]. It hasbeen noted that these principles wereused by the signal-processing communitybefore being formalized in research, andhave, therefore, already been demon-strated to be beneficial to softwareengineering [Lee and Parks 1995].

5. RECENT DEVELOPMENTS IN DATAFLOWPROGRAMMING LANGUAGES

5.1. Introduction

The last major reviews of dataflow pro-gramming languages were Hils [1992] andWhiting and Pascoe [1994]. In the decade



since then, the field of dataflow has ex-panded and diverged to include manydisparate areas of research. However, thefocus of this section is strictly on program-ming languages that are based upon thedataflow execution model. Indeed, thereare some languages that have the appear-ance of dataflow, but upon examination, itis clear that they are sufficiently differentfrom the pure dataflow execution model tomake such a label questionable, for exam-ple, JavaBeans.

Since the majority of developments indataflow programming languages in thepast decade have been in the field of vi-sual programming languages, this sec-tion also concentrates on visual program-ming languages. It should be stressed thatthe textual dataflow languages detailed inprevious sections still exist and are beingdeveloped, although most of the currentresearch in that area is in the field of hard-ware and compilation technology. Sincethe emphasis in this article is on softwareengineering rather than hardware, theseissues are beyond the scope of this section.Hardware issues are mentioned only inso-far as they have affected the developmentof dataflow languages.

As has already been outlined in the pre-vious section, the major development inthe past 10 years in dataflow has beenthe move away from fine-grained paral-lelism toward a more coarse-grained ap-proach. These approaches ranged in con-cept from adding limited von Neumannhardware to dataflow architectures, torunning dataflow programs in a multi-threaded manner on machines that werelargely von Neumann in nature.

To some extent, dataflow languagesevolved to meet these new challenges. Agreater emphasis was placed upon com-piling dataflow programs into a set ofsequential threads that were themselvesexecuted using the dataflow firing rules.However, these changes did not have a ma-jor effect upon the languages themselveswhose underlying semantics did not haveto change.

From a software engineering perspec-tive, however, the major development indataflow in the past 15 years has been the

growth of dataflow visual programminglanguages (DFVPLs). Although the the-ory behind DFVPLs has been in existencefor many years, it is only the availabilityof cheap graphical hardware in the 1990sthat has made it a practical and fruitfularea of research.

Investigations of DFVPLs have indi-cated many solutions to existing problemsin software engineering, a point whichwill be expanded upon below. It has alsoled to the introduction of new problemsand challenges, particularly those associ-ated with visual programming languagesin general [Whitley 1997], as well as con-tinuing problems, such as the represen-tation of data structures and control-flowstructures [Auguston and Delgado 1997;Ghittori et al. 1998; Mosconi and Porta2000]. Research has been fairly intense inthe past decade, and it is the subject ofthis section to identify some of the maintrends in dataflow programming over thisperiod.

5.2. The Development of Dataflow VisualProgramming Languages

In Section 3, textual dataflow languageswere discussed, and much of the researchinto dataflow hardware utilizes these tex-tual languages. The “machine” language ofprograms designed to be run on dataflowhardware architectures is the dataflowgraph. Most textual dataflow languageswere translated into these graphs in orderto be scheduled on the dataflow machine.

However, early on it was realized thatthese graphs could have advantages inthemselves for the programmer [Davis1974; Davis and Keller 1982]. Graphsallow easy communication of ideas tonovices, allowing much more productivemeetings between the developer and thecustomer [Baroth and Hartsough 1995;Morrison 1994; Shurr 1997]. In addition, arange of research into VPLs has indicatedthe existence of significant advantages ina visual syntax [Green and Petre 1996],for example, dynamic syntax and visual-ization [Hils 1992; Shizuki et al. 2000].The fact that several dataflow environ-ments have been the basis of successful


18 Johnston et al.

commercial products adds weight to thiscase [Baroth and Hartsough 1995]. Fi-nally, research has shown that most devel-opers naturally think in terms of dataflowin the design phase, and DFVPLs re-move the paradigm shift that is forced ona programmer when entering the codingphase. Indeed, DFVPLs arguably removethis distinction altogether [Baroth andHartsough 1995; Iwata and Terada 1995].

Researchers published papers onDFVPLs intermittently in the 1980s.Their ideas were intriguing and showedgreat promise, but were restricted by theexpense and low diffusion of graphicalhardware and pointing devices. Davisand Keller [1982] recognized the nowuniversally accepted trend toward moregraphically based computer systems,and made the argument that textuallanguages could be completely replacedby graphical ones in the future. Althoughthis prediction has not fully come to pass,they judiciously proposed that humanengineering rather than concurrentexecution would become the principalmotivation for developing dataflow visualprogramming languages, a motivationthat has indeed been at the fore of morerecent DFVPL research.

5.2.1. Early Dataflow Visual ProgrammingLanguages. In the 1970s, Davis [1974,1979] devised Data-Driven Nets (DDNs), agraphical programming concept that wasarguably the first dataflow visual lan-guage (as opposed to a graph used purelyfor representation). In DDN, programs arerepresented as a cyclic dataflow graphwith typed data items flowing along thearcs which are FIFO queues. The pro-gram is stored in a file as a parenthe-sized character string, but displayed as agraph. The language operates at a verylow level and, in fact, Davis [1978] com-mented that it was not the intention thatanyone should program directly in DDNs.Nevertheless, they illustrated key con-cepts such as the feasibility of providingiteration, procedure calls, and conditionalexecution without the use of a textuallanguage.

By the early 1980s, Davis had devel-oped a more practical, higher-level DFVPLknown as GPL (Graphical ProgrammingLanguage). Davis and Lowder [1981] con-tended that text-based programming lan-guages lacked intuitive clarity and pro-posed going further than using graphsas a design aid by creating an environ-ment in which the program is a graph.GPL was also an attempt to create ahigher-level version of DDNs [Davis 1979;Whiting and Pascoe 1994]. In the GPL en-vironment, every node in the graph waseither an atomic node or could be ex-panded to reveal a sub-graph, therebyproviding structured programming withtop-down development. These subgraphscould be defined recursively. Arcs, in thegraph were typed and the whole environ-ment had facilities for debugging, visu-alization and text-based programming ifdesired. The lack of suitable graphicalhardware for their system was the mainreason for a lack of rapid development ofthese concepts.

In the early 1980s, researchers Kellerand Yen [1981] developed FGL, indepen-dently from Davis. FGL stands for Func-tion Graph Language, and was born fromthe same concept of developing dataflowgraphs directly. Unlike the token-baseddataflow model of GPL, FGL was basedaround the structure model, of whichKeller was a proponent [Davis and Keller1982] (see Section 2.1.2). Under thismodel, data is grouped into a single struc-ture on each arc rather than flowingaround the system. In other regards, FGLwas similar to GPL in its support for top-down stepwise refinement. The relativeadvantages and disadvantages of GPL andFGL mirror those of the token-flow modeland structure model, respectively.

Shortly afterwards, the Grunch systemwas developed by de Jong et al. [1982], thesame researchers who created the Cajoletextual dataflow language [Hankin andGlaser 1981]. While not a programminglanguage in the proper sense, it was agraphical overlay for Cajole that allowedthe developer to graphically express adataflow program using stepwise refine-ment, and then use the tool to convert



the graph into Cajole. The actual con-version was performed by an underlyingtool called Crunch. The development ofGrunch supported the claims of Davis andKeller [1982] that software engineeringcould be as much a motivation for pursu-ing graphical dataflow as the pursuit ofefficient parallelism.

5.2.2. More Recent Dataflow Visual Program-ming Languages. Interestingly, from themid-1980s on, further development ofDFVPLs often came from different sourcesthan direct research into dataflow. Indeed,industry played a part in this phase of de-velopment. The most common source wassignal- and image-processing, which lendsitself particularly well to a dataflow ap-proach [Buck and Lee 1995]. Therefore,many DFVPLs were produced to solvespecific problems and utilized dataflowbecause it provided the best solution tothe problem. As Hils [1992] commented,DFVPLs in this period were most success-ful in narrow application domains and indomains where data manipulation is theforemost task.

Hils [1992] provided details of 15 lan-guages developed in the 1980s and veryearly 1990s that could be classed asDFVPLs. In order to avoid repetition,only two examples of these are discussedhere. NL, a significant language that ap-peared after Hils wrote his paper, is alsodescribed.

LabView is a well-known DFVPL devel-oped in the mid-1980s to allow the con-struction of “virtual” instruments for dataanalysis in laboratories. As such, it wasintended for use by people who were notthemselves professional programmers. Aprogram in LabView is constructed by con-necting together predefined functions, dis-played as boxes with icons, using arcs fordata paths. Each program also has a vi-sual interface to allow the design of the vir-tual instrument. Components that have avisual representation appear both in theinterface and the program, whereas func-tions only appear in the program window.The whole program is executed accord-ing to the dataflow firing rules. LabView

makes the programming experience lesscumbersome by providing iterative con-structs and a form of stepwise refinementwhereby programmers can produce theirown function nodes.

Empirical evidence reported by theJet Propulsion Laboratory [Baroth andHartsough 1995] has shown a very favor-able experience with LabView when usedfor a large project, compared to develop-ing the same system in C. In particular,they found that the DFVPL led to a sig-nificantly faster development time than C,mainly due to the increased communica-tion facilitated by the visual syntax. Anexample of a program written in LabViewis shown in Figure 5. As well as itsdemonstrated and continuing industrialsuccesses, LabView has proved particu-larly popular with researchers [Ghittoriet al. 1998; Green and Petre 1996].

ProGraph was a more general-purposeDFVPL than LabView, and involvedcombining the principles of dataflowwith object-oriented programming. Themethods of each object are defined us-ing dataflow diagrams. Like LabView,ProGraph includes iterative constructsand permits procedural abstraction bycondensing a graph into a single node.ProGraph has also been used as a subjectin research [Cox and Smedley 1996;Green and Petre 1996; Mosconi and Porta2000]. Example screenshots of ProGraphprograms can be found in Mosconi andPorta [2000].

In the mid 1990s, the language NL wasdeveloped by Harvey and Morris [1993,1996], along with a supporting program-ming environment. It is fully based on thedataflow model of execution. NL has anextended typing system, whereby arrayscan behave as arbitrarily long lists, to thepoint of being infinite. It provides an in-genious method of control flow, throughcombined use of “block” and “guard” nodes.For example, a guard node may contain acondition that, if evaluated to true, causesits associated block node to be executed.Sequences of guard nodes can be created,and once one guard has been executed, allothers are ignored. This has the advan-tage of reducing screen clutter and making


20 Johnston et al.

Fig. 5. Example program in LabView designed to find the real roots of a quadratic equation.

the flow of control more explicit. Loops aresupported by a related method.

The NL environment reported inHarvey and Morris [1996] features avisual debugger. Screenshots of typicalNL programs can also be found in thisarticle. Programmers sees their programin the same way that they developed itand can choose to step one firing at atime, or use breakpoints. A node thatis firing is highlighted, and placing thecursor over a port allows them to examineand change values. When it comes toloops, a slider allows the programmersto select any of the iterations that aretaking place and examine them in anyorder.

5.2.3. Dataflow as a Coordination Language.Gelernter and Carriero [1992] emphasizedthe concept of the coordination language,that is, the concept that programming con-sists of two tasks, not one: computation,which specifies what is to be done, andcoordination, which specifies how the com-putations are to proceed. They argued thatthe need for such a distinction was becom-ing more necessary with the advent of dis-tributed and heterogeneous computer sys-tems. The proposal was for a separation

of the computation language and the co-ordination language.

Dataflow researchers have taken thisidea on board. Indeed, since dataflowgraphs explicitly express the relationshipsbetween computations, it is clear thatdataflow is a natural coordination lan-guage. While few researchers have goneso far as to create an entirely indepen-dent general-purpose co-ordination lan-guage based on dataflow ideas, many haveproduced DFVPLs that strongly displaythe distinction. For example, the VipersDFVPL [Bernini and Mosconi 1994] is acoordination language where the nodes inthe graph are expressed using the lan-guage Tcl.

Morrison’s [1994] flow-based program-ming concept, while it does not strictlyobey the rules of dataflow, describes a sys-tem where nodes are built in arbitraryprogramming languages which the pro-grammer arranges using a single networkediting environment. Morrison [1994]reported empirical evidence that appearsto support his assertion that this methodis practical in real-world situations.

An excellent example of such a systemis Granular Lucid (GLU) which was devel-oped by Jagannathan [1995]. It is basedupon Lucid, with the key addition that



functions are defined in a foreign, proba-bly sequential, language such as C. Datatypes are also of a foreign format. SinceLucid itself is a textual dataflow language,GLU allows a much more coarse-grainedapproach to dataflow by the programmer.Instead of primitive operations being ex-ecuted in a fine-grained manner, this al-lows the rules of dataflow to be applied toa much coarser granularity. Jagannathan[1995] went on to show how this degreeof granularity achieves performance simi-lar to conventional parallel languages andconcluded that using dataflow program-ming languages to develop applicationsfor conventional parallel processors isfeasible.

All of the above-mentioned languagesare examples of how dataflow program-ming may be moved to a higher levelof abstraction. For example, a languagein which entire functions are enclosedwithin a node could be envisaged, whilethe nodes themselves are executed, usingthe dataflow semantics. This would be adevelopment of the ideas put forward byBernini and Mosconi [1994] and by Rasureand Williams [1991].

With the recent trend toward het-erogeneous distributed systems andcomponent-based programming, it isbelieved that thinking of dataflow as a co-ordination language has much merit andone that deserves further investigation.

5.3. Assessment of Visual DataflowProgramming Environments

The power of any visual programminglanguage depends more heavily upon itsenvironment than its text-based counter-parts. The ease with which tasks can beperformed has a large bearing on howit compares to other languages. Thosewho have used DFVPLs in industry havecommented that the visual nature is anessential component of the language, notsimply an interface, and that without thevisualization tools offered by the environ-ment, DFVPLs would have limited use[Baroth and Hartsough 1995].

In keeping with this fact, the trendin the late 1990s has been toward de-

veloping programming environments intandem with the DFVLPs that they use.Indeed, so tightly have the two becomethat it has become difficult to distinguishwhere the language ends and the envi-ronment begins. Therefore, this sectionnecessarily overlaps with language issues.Many of the advantages of DFVPLs are ad-vantages of their environments as much asof the language.

Burnett et al. [1995] discussed what isneeded in order to scale up a visual pro-gramming language to the point of be-ing a practical proposition for a sizeablereal world project. They came up with alist of four things that VPLS are tryingto achieve. These are the reduction in keyconcepts, such as pointers; a more concreteprogramming experience, such as explor-ing data visually; explicit definitions of re-lationships between tasks; and immediatevisual feedback.

DFVPLs have the potential to achieveall of these to some degree. The dataflowgraph itself is an ideal example of anexplicitly defined relationship, and it istrue that they have a smaller set of keyconcepts than their textual counterparts.A number of the languages reviewed byHils [1992] feature a high degree of live-ness, that is, immediate visual feedback,with one, VIVA [Tanimoto 1990], allow-ing programmers to dynamically edit theprograms while it is running visually infront of them. Finally, the visual debug-ger in Harvey’s NL environment [Harveyand Morris 1996] is a good example bothof the immediate feedback of informationand the visual exploration of data.

Of course, it is more difficult to mea-sure how well a DFVPL meets the crite-ria that it sets out to achieve. There arefew metrics available in literature at thisstage, although Kiper et al. [1997] offeredone set of subjective metrics for measuringVPLs in general. Their criteria includedits scalability, its ease of comprehension,gauging the degree of visual nature of thelanguage, its functionality, and its supportfor the paradigm. The first of these points,that of scalability, has been answered tosome degree by Burnett’s work, mentionedabove [Burnett et al. 1995].


22 Johnston et al.

An attempt to measure the compre-hension of dataflow languages was madeby Green and Petre [1996]. They stud-ied ProGraph and LabView at length, andconcluded that they had clear advantages.They drew the following interesting con-clusions regarding the current state ofDFVPLs:

—that DFVPLs allow the developer to pro-ceed with design and implementation intheir own order, thus making the designprocess freer and easier;

—that secondary notation could be uti-lized much more than it currently was;

—that more work needed to be con-ducted on incorporating control-flowconstructs;

—that the effectiveness of program ed-itors remained to be investigated inliterature;

—that the problem of real estate was notas major as many assume it to be.

Further feedback on what is neededin DFVPLs was provided by Baroth andHartsough [1995]. Having used a DFVPLfor a real-world project, they concludedthat the advantages offered lie more to-ward the design end of the software lifecy-cle, and less in the later stages of coding.They found increased communication be-tween developer and customer, comment-ing, “We usually program together withthe customer at the terminal, and theyfollow the data flow diagrams enough tomake suggestions or corrections in theflow of the code. It is difficult to imaginea similar situation using text-based code”[Baroth and Hartsough 1995, p. 26]. Thedevelopment time improvement in thiscase was a factor of 4.

By contrast, Baroth and Hartsough[1995] commented that the provisionof software libraries, while speeding upcoding, is merely a case of “who can typefaster,” and is not an advantage in itself.And so, the issue of the provision of alibrary of nodes is not a major one forDFVPLs. Already, a DFVPL can be usedwith a very primitive set of nodes and theprovision of nodes that can be built up

from these primitives is really an issue forthe vendor of the programming environ-ment, not for academia.

A point that Baroth and Hartsough[1995] were keen to stress was that vi-sualization, and animation in particular,is absolutely essential to making the tooluseful. Indeed, they went so far as tocomment that “the graphics description ofthe system without the animation wouldnot be much more than a CASE tool witha code generator” [Baroth and Hartsough1995, p. 28].

A final point made by Baroth andHartsough [1995] was that the bound-aries between the requirements, design,and coding phases of the software lifecyclecollapse and blend into one another. Thisappears to be both an advantage and a dis-advantage. It is a problem in that the ex-isting methodologies in place were unableto support the tool and this led to an inabil-ity to assess the progress in the project.On the other hand, the single phase al-lows the customer to be involved at allstages, reducing the prospect for expen-sive mistakes, and also reducing develop-ment time.

It is the previously mentioned empha-sis on the design phase that prompted thedevelopment of Visual Design Patterns(VDPs) by Shizuki et al. [2000; Toyodaet al. 1997]. Under the VDP approach, theuser is equipped with generic design pat-terns of common task layouts. Developerschoose a VDP to suit their needs and theninsert specific components into the holes inthe pattern in order to produce an actualimplementation. The concept has been in-troduced into the KLIEG environment anddemonstrated in the literature. In theirmore recent paper, Shizuki et al. [2000].extended the idea to include the possibilitythat the use of VDPs could help to focus asmart environment on the specific aspectsof dataflow execution that a developer islikely to be interested in.

Animation is an important concept thatwas highlighted above. There is a sig-nificant difference between viewing aprogram graphically, and viewing it dy-namically. The animation of executing



dataflow programs is an exciting topic,but one in which research has only re-cently been undertaken in detail. A goodexample is Shizuki et al. [2000], which ex-plored how a large program can be ani-mated for a programmer. This addressedthe problems of how to view multiple lay-ers at once, how to view different areasof the program at once, how to change fo-cus rapidly so as to avoid loss of concentra-tion, and how to create sufficiently smoothanimation that will not appear disjointedto the developer. The solution proposedis a smart, multifocal, fisheye algorithm.Much research deserves to be done in thisarea.

On the basis of this discussion, the fol-lowing conclusions concerning DataflowVisual Programming Environment can bedrawn:

—In a DFVPL there is a blur inthe distinction between language andenvironment.

—In addition, DFVPLs tend to sig-nificantly blur the distinctions be-tween the requirements, design, cod-ing, and testing phases of the softwarelifecycle.

—This blurring offers the opportunity forrapid prototyping.

—The design phase benefits the mostfrom the use of DFVPLs over textuallanguages.

—The animation offered by a DFVPL en-vironment is vitally important to itsusefulness.

—The dataflow semantics of DFVPLs areintuitive for nonprogrammers to un-derstand and thus improve communi-cation between the customer and thedeveloper.

—The library of functions included witha DFVPL is not a major factor inproductivity.

—Key areas requiring work include theuse of secondary notation, and control-flow constructs.

6. OPEN ISSUES IN DATAFLOWPROGRAMMING

Dataflow programming is an active areaof research, and many problems remainopen. Four of these issues are discussedin more detail in this section:

—the provision of iteration in textualdataflow languages,

—iteration structures in DFVPLs,—the use of data structures,—nondeterminism.

6.1. Iteration in Textual Dataflow Languages

Most dataflow programming languagesprovide loops of some form, but the wayin which loops are expressed as a dataflowgraph is quite different from most otherrepresentations of iteration. The prob-lem arises because iteration does not fitneatly into the functional paradigm, asit involves repeated assignment to aloop variable and sequential processing.Nevertheless, most dataflow researchersrecognized that programmers’ demandsmade it necessary to provide iteration[Ackerman 1982] and worked on waysto make it mathematically respectable[Ashcroft and Wadge 1977]. Ways of mak-ing it efficient were also studied [Ningand Gao 1991]. It should be noted thatmany dataflow languages provide itera-tion through tail-recursion. However, asthis is usual practice in functional lan-guages, this section deals specifically withthe more explicit iterative constructs.

The exact syntax of the various solu-tions offered differed, but they were allfundamentally the same. The idea wasto think of the body of an iteration asbeing executed in an environment wherethe loop variable had a certain value thatremained the same throughout the itera-tion. Thus, a single pass of the loop canbe regarded as a set of definitions like anyother. The loop variable is updated by us-ing an identifier such as “NEW” to refer tothe value that the loop variable will haveon the next iteration. For example,

NEW X = X + 1;


24 Johnston et al.

Fig. 6. Dataflow graph representing the factorial program.

As the value of X has not actually beenchanged by this statement, this is a math-ematically acceptable way of representingiteration. When the loop has completedthe iteration, the value of NEW X is as-signed to X, but again, this is acceptablesince all that is required is that the valueof X remain unchanged during the sin-gle iteration. Some languages use an “old”keyword to achieve the same effect.

A piece of code to calculate factorial(N)by iteration, when translated into thefunctional loop favored by dataflow pro-gramming languages, looks like this:

LOOP WITH i = N, fact = 1NEW fact = fact ∗ i;NEW i = i − 1;

WHILE i > 1;

In this code, the values of “fact” and “i” aredefined functionally, using the loop. Theyare modified using the keyword “NEW.”Note that the definitions of “NEW fact”and “NEW i” can be placed in any order.If the definition of “NEW i” were placedfirst, the definition of “NEW fact” wouldstill be valid because the original value of iis unchanged until the end of the iteration.Note also that the value “fact” must be de-

clared as a loop variable in the dataflowversion of the loop because that is theonly way that a variable can be assignedmultiple times in the manner required byiteration.

While this code definitely looks differ-ent from the imperative example, it does,nevertheless, retain a strong imperativefeel and could be used more intuitivelyby programmers when compared to tail-recursion.

Of course, this code has to be translatedinto a dataflow graph before it can be ex-ecuted. While a loop in a dataflow graphcan look complicated, most loops can becoded in the same way. Figure 6 shows adataflow graph that could result from theabove dataflow code example.

It cannot be denied that this repre-sentation is much less succinct than thetext-based loop. However, the point is notthat a loop can be drawn directly as agraph, but that the text-based loop canbe converted into a well-behaved dataflowgraph. Few dataflow researchers would ex-pect any programmer to manually gener-ate the graph shown in Figure 6. This alsoillustrates one of the failings of earlygraphical dataflow languages. However,as we shall note in the next section, recent



development in dataflow research permitprograms to be specified graphically with-out this level of detail.

In Figure 6, the rectangles marked“Switch” and “Merge” operate as explainedin Section 2.1.1. The “Delay” gate sim-ply waits until data appears at both in-puts before outputting the data on the leftarc and discarding the data on the rightarc. This acts as a trigger, preventing theloop from repeatedly executing ad infini-tum. With the Delay gate, a single tokenpassed down the trigger arc will cause oneexecution of the loop. The squares with“1” in them are constants, that repeat-edly generate tokens with the value “1.”The circles are nodes that perform opera-tions. These operations produce either nu-merical or Boolean results. The small greycircles labeled “F” signify tokens that aredefined to be present on the given arcswhen the program first activates. Threehorizontal parallel lines denote a sink,which destroys any tokens that fall into it.A small open square at a crossing of twoarcs indicates that the arcs are joined. Inall other cases, arcs pass over each otherwithout being joined.

If executed under the pure token-baseddataflow model, with N = 3, the 32 sepa-rate firings necessary to complete the exe-cution are performed in 14 time units, withparallelism in each time unit of either twoor three instructions. The left-hand side ofthe graph produces a sequence of tokensrepresenting the counter “i,” starting at 1.The right-hand side produces a series of to-kens representing the accumulating facto-rial by multiplying the previous factorialtoken by a token from the first sequenceeach time. The node in the center of thegraph halts the feedback once the valueof i ≤ N becomes false, and the Switchgates are used to output the completed val-ues. The value of “i” is discarded, while thevalue of the factorial is sent out of this por-tion of the graph.

When executed under the pure token-based dataflow model, as above, the graphexhibits some pipelining within one in-stance of the loop, as the next iterationcan begin before the previous one hascompleted. However, the necessary condi-

tional check delays the execution of theSwitch nodes that could otherwise beginto execute sooner. When the situation ofseveral loops being executed at once is con-sidered, that is, by several triggers arriv-ing simultaneously, the pure model per-mits very little overlap of separate loopinstances: in this case, the maximum over-lap is under 10%. This is because the to-ken that will begin the next loop is delayedby the Merge gate until a false token ar-rives, and this only occurs when the pre-vious loop has completed.

When executed under the dynamicmodel, the loop does not provide any morepipelining of iterations within a loop, butit does provide excellent overlap of sepa-rate iterations. In fact, they can occur si-multaneously and independently. Underthe dynamic model, the “false” tokens thatare initially placed at the two Merge gateswill be present initially for each separateinstance of the loop, rather than havingto be generated by the previous loop asit completes. It should also be noted thatother loops, such as those which populatearrays, do not have the pipelining prob-lems mentioned above.

As illustrated above, some loops indataflow graphs have the potential tolimit concurrency. However, the use of al-ternative models of execution can limitthis restriction. Although it is not to-tally natural in functional languages, it-eration has been accepted as necessaryby most researchers. Indeed, Whiting andPascoe [1994, p. 53] commented that “theintroduction of this form of loop con-struct . . . was responsible for much of theacceptance of data-flow languages.” Theefficient execution of dataflow loops hasbeen a subject of active and ongoing re-search [Bic et al. 1995; Ning and Gao 1991;Yu and D’Hollander 2001].

6.2. Iteration in Dataflow VisualProgramming Languages

Although iteration remains an open ques-tion in DFVPLs as well as textual dataflowlanguages, it is a different kind of prob-lem. Here the problem is how to express arepetitive structure in a graphical model


26 Johnston et al.

that does not naturally allow such struc-tures. Figure 6 shows how a loop looksunder the pure dataflow model. Few pro-grammers would wish to construct such agraph, and, if they did, it would be un-clear and error-prone. It has long beenrecognized that a practical DFVPL mustprovide a better way to support iteration.The question has been what constructs aremost appropriate for expressing iteration.It should also be noted that iteration ismerely an example of the wider issue ofhow to express control-flow constraints inDFVPLs. However, since iteration is ar-guably the most important and heavilyresearched problem, this section concen-trates on it.

A key recent article on this topic wasMosconi and Porta [2000], and we do notintend to reproduce their review. Instead,each of five examples of iteration con-structs will be described briefly.

—Show-And-Tell. Show-and-Tell [Kimuraand McLain 1986] was an early dataflowvisual language designed for children.In its approach to iteration, a specialnode is used to enclose an area of codethat is to be executed iteratively. Eachloop box has what is known as a consis-tency check. Data can only flow througha node if it is consistent. If the consis-tency check evaluates to false, the nodebecomes inconsistent, and execution ofthe loop stops. The loop has the samenumber of inputs, as outputs and datais fed back from the outputs into the in-puts, as long as the box is consistent.When it becomes inconsistent, the datais ejected to the rest of the graph.

For example, a loop might contain oneinput that identifies the number of it-erations required. This value is decre-mented and sent to the output duringeach iteration. The consistency checkis that this value is greater than zero.Thus, when the iteration count reacheszero, the loop stops executing. Screen-shots of Show-and-Tell loops can befound in Mosconi and Porta [2000].

—LabView. LabView, a commercial prod-uct, has two kinds of loop, a FOR loopand a WHILE loop [LabView 2000]. Like

Show-and-Tell, a FOR loop is a specialnode that encloses all of the nodes to beexecuted iteratively. Unlike Show-and-Tell, it has an additional input port thatspecifies how many times the loop is torun. All other values that are outputports conceptually reenter on identicalinput ports. Another port visible only in-side the loop specifies the current valueof the loop variable.

The WHILE loop operates in a sim-ilar way, except that it does not havethe loop variable. Instead, it has a portonly visible inside the loop that termi-nates after the current iteration once itreceives a value of “false.” A constructunique to LabView allows the timingsof the loop to be specified, for example,loop every 250 ms. This is due to itsapplication of reading scientific instru-ments. A LabView program is shown inFigure 5. Further screenshots can befound in Mosconi and Porta [2000].

—Prograph. In Prograph [Cox et al. 1989],any user-defined node that has the samenumber and type of inputs as outputscan be deemed to be a loop. Its iconchanges to illustrate this fact. Prographprovides a special “terminate” node foruse within a loop. When the conditionspecified within the terminate node issatisfied, the iteration is terminated af-ter the current iteration is complete.

—Cantata. Cantata [Rasure and Williams1991] is a coarse-grained language inwhich nodes contain entire functions,rather than just a primitive operation.Its approach is to conceal the entire loopwithin one node. Each input is desig-nated a name by the programmer, whoalso specifies either a loop variable andbounds, or a WHILE-condition, usingthe names. The programmer then setsup a series of assignments that are totake place within each loop. The nodethen executes the loop internally.

Note that this is far removed frompure dataflow philosophy. For exam-ple, a loop assignment may containthe expression j = j + 1, a statementwhich traditionally makes no sensein a dataflow language. Examples of



Cantata programs can be found inMosconi and Porta [2000].

—VEE. In contrast with Cantata, loops inVEE [Helsel 1994] are expressed mostclosely to the pure dataflow model, thatis, through cycles in the graph. However,the model has been augmented with anumber of additional nodes in order tosimplify the appearance of the cycles. Ina FOR loop, a special FOR node gener-ates a series of indexes between a rangethat are queued. The programmer doesnot need to worry about incrementingand feeding back the loop variable, beingfree instead to concentrate on the val-ues being calculated. A WHILE loop canbe set up by using three related nodes.The UNTIL BREAK node repeatedly ac-tivates the graph it is connected to, un-til the graph activates a related BREAKnode which halts the repetition. Data ar-riving instead at the NEXT node trig-gers the next iteration.

Mosconi and Porta [2000] concludedtheir paper by proposing a syntax that isconsistent with the pure dataflow model.They were keen to stress that they werenot proposing their syntax for actual use,but to prove that practical iteration ispossible without sacrificing the pure se-mantics of the model. Their loop system,implemented as part of the Vipers envi-ronment [Ghittori et al. 1998], includescycles, but is simplified by the use of en-abling signals. They also demonstrateda way to collapse an iteration into asingle node without sacrificing the puremodel.

All of these approaches have advantagesand disadvantages. Some, such as Can-tata, introduce imperative structures thatare inconsistent with dataflow, althoughCantata also offers the simplest loops interms of visual syntax. Others, such asVEE, involve relatively complex graphs.All of them suffer from the inability to dy-namically express the concept of a repet-itive loop with a static icon. The wholearea of control-flow constructs, and itera-tion in particular, remains an open topic inDFVPLs.

6.3. Data Structures

One of the key issues in the drive for an ef-ficient implementation of dataflow is thatof data structures. Whiting and Pascoe[1994] commented that “data structuressit uneasily within the data-flow model”(see also Treleaven et al. [1982] and Veen[1986]). However, they went on to note thatmuch research has been undertaken inthis area and that a number of quite suc-cessful solutions have been proposed, mostnotably I-structures [Arvind et al. 1989].

The “pure” token model of dataflowstates that all data is represented by val-ues that, once created, cannot be modified.These values flow around the dataflowgraph on tokens and are absorbed bynodes. If a node wishes to modify thisvalue, it creates a new token, containingnew data which is identical to the origi-nal data, except for the element that hadto be altered. Some of the earliest dataflowlanguages that had support for data struc-tures worked in this way [Davis 1979]. Ifthis way of treating data as values ratherthan variables were not part of the to-ken model of dataflow, then the single-assignment rule would have been violatedand thus the data-dependent scheduling ofthe entire graph would be compromised.

While this conceptual view of data struc-tures is perfectly fine for the theoreti-cal study of dataflow, and perhaps evenfor dataflow graphs that deal only withprimitive data types, this approach isclearly unsatisfactory for graphs that re-quire the use of data structures. With theera of structured programming, followedby the era of object-oriented programming,the idea of software development with-out the use of data structures is virtu-ally incomprehensible. Thus, any practicalimplementation of dataflow must includean efficient way of providing data struc-tures, although it should be stated thatsome languages designed for research pur-poses solved the problem by not providingdata structures at all [Hankin and Glaser1981].

6.3.1. Dennis’s Method. Dennis [1974]was the first to provide realistic data


28 Johnston et al.

Fig. 7. Showing the effect on Dennis’s data heapof modifying a value.

structures in a dataflow context by propos-ing that the tokens in the dataflow pro-gram hold not the data itself, but rathera pointer to the data (see also Davis andKeller [1982]). He devised a memory heapin the form of a finite, acyclic, directedgraph where each node represents eitheran elementary value or a structured valuewhich behaves much like an indexed ar-ray. Each element of a structured value is,in turn, a node that represents either anelementary value or a structured value.The pointers in the data tokens refer toone of these nodes and a reference countis maintained. A node which is no longerreferred to, either directly or indirectly,in the graph is removed by an implicitgarbage collector.

Dennis [1974] went on to show thatimplementation is possible without needof copying arbitrarily complex values. AsDennis’s dataflow programs were alwaysfunctional, it was necessary that modify-ing a value should result in a new value,without changing the original. Wheneveran elementary value is modified, a newnode is simply added to the heap. When-ever a structured value is modified, andthere is more than one reference to thevalue, a new root node is added to the heap,pointing to the same values as the orig-inal root node, with the exception of theone value that was to have been modifiedfor which a new node is created. This isillustrated by Figure 7, which shows theeffect on Dennis’s data heap when a valueA, which represents the array [a, b, c],is modified. The second element is modi-fied to create a new value B, which rep-resents the array [a, e, c]. In the memory

heap, value B retains references to all thedata that is not modified, thus saving timeby not copying the entire data structure.Meanwhile, value A remains unmodified,preserving the functional semantics of themodel.

This method prevents the unboundedcopying of arbitrarily complex values. Italso permits the sharing of identical dataelements which saves memory. However, itis not an ideal solution for all situations.For example, if the values in a 100-elementarray are being modified sequentially by aloop, this solution would require making100 new data structures in the process,notwithstanding the fact that they are notcopying the entire array each time. A goodcompiler could detect such a loop and pre-vent needless copying such as this. Ex-cessive overhead in the Dennis approachwas also examined and reported by Gajskiet al. [1982].

A second problem, and one which be-came more evident as research progressed[Ackerman 1982], was that the use of datastructures can reduce parallelism in adataflow program due to the long delay be-tween creating the structure, and all partsof it being completed. To use Ackerman’s[1982] example, consider a dataflow pro-gram that has two main sections. The firstcreates a 100-element array and populatesit, one element at a time. The second takesthe array and reads the elements, one ata time. In this case, the second part can-not begin to execute until the first part iscomplete, even though it could be readingelement 1 while the first part of the pro-gram is writing element 2, and so on. Inthis case, a program that could conceiv-ably execute in 101 time units, takes 200time units to complete. This delay provedto be frequently unnecessary, and led tothe development of I-structures.

6.3.2. I-Structures. To overcome thisproblem, Arvind and Thomas [1980]proposed a system that they calledI-structures. They observed that theproblem with Dennis’s approach was thatit imposed too strict a control structure



on the computations that filled in thecomponents of the data structure [Arvindet al. 1989]. Ideally, what was neededwas some way of allowing more flex-ible access to the data structures butwhich—crucially—would not destroy thefunctional semantics of dataflow.

I-structures are related to lazy evalua-tion. A data structure is created in mem-ory, but its constituent fields are left blank.Each field can either store a value or be“undefined.” A value can only be stored ina field that is currently undefined. ThusI-structures obey the single-assignmentrule. Any attempt to read from an unde-fined field is deferred until that value be-comes available, that is, until a value isassigned to it.

By following this set of rules, a datastructure can be “transmitted” to the restof the program as soon as it is created,while the sender continues to populate thefields of that structure. Meanwhile, the re-ceiver can begin to read from the struc-ture. Referring again to Ackerman’s [1982]example, this would dramatically improvethe performance of some programs. Al-though they are somewhat opposed to thepurity of dataflow, I-Structures have beenwidely adopted [Arvind et al. 1988; Culleret al. 1995; Keller 1985].

However, while I-structures do solveproblems of unnecessary delays in func-tional dataflow programs, they do not ad-dress the initial problem of copying datastructures in order to modify them. Gajskiet al. [1982] pointed out other issues re-lated to the overhead of storing and ful-filling deferred reads in this approach, al-though Arvind and Culler [1986] arguedthat this overhead is small and easily out-weighed by the benefits. This latter viewappears to be supported by the experi-ments of Arvind et al. [1988].

6.3.3. Hybrid Structures. Hurson et al.[1989] examined both Dennis’s copyingstrategy and the later I-structures. As dis-cussed above, they found that the for-mer had high-overhead issues and wastedpotential parallelism, while the latter

wasted space as it was unable to sharecommon substructures. They claimed thattheir proposed hybrid structures carefullycombined the advantages brought forthby both copying and sharing. To demon-strate their proposed structure, they usedarrays, although the method can be ap-plied to any form of data structure.

Their method represents each array asan array template (for full details, seeHurson et al. [1989]). The array templatehas a reference count and can either referto an original array or a modified array.In the case of an original array, the tem-plate points to a sequential area of mem-ory that contains the elements of the array.Whenever an element of the array is “mod-ified,” a new “modified” array template iscreated to represent the new array. It con-tains an area of memory that representsthe new array, but with only the modi-fied value filled in. The other values aremarked as absent and a link is providedback to the original array. An attempt toread from the new array will either re-turn a modified value or, if the value is notthere, the link to the original array willbe followed and the value retrieved fromthere.

If another value is modified in the newarray, and its reference count remains 1,it can be modified in situ without hav-ing to create a new array template. Inorder to prevent copying the entire arrayeach time, a new array is required; hybridstructures allow large arrays to be brokenup into equally sized blocks, each storinga certain number of elements. Because theblocks are of identical size, looking for anelement within them can be achieved inconstant time. Blocks themselves have ref-erence counts, allowing for sharing of sub-portions of arrays.

Experiments reported in the same paper[Hurson et al. 1989] suggest that hybridstructures lead to improvements in bothperformance and storage over the copyingmethod of Dennis [1974] and I-structures,although it does contain a certain elementof overhead.

While the three approaches outlinedabove, and their derivatives, have resolved


30 Johnston et al.

many of the problems related to effi-ciently implementing data structures inthe dataflow model, those problems nev-ertheless remain open issues. A briefoverview of data structures and dataflowwas given in Lee and Hurson [1993]. Ef-forts to reduce unnecessary copying andto reduce wasted memory from needlessduplication, and the desire not to re-duce parallelism when implementing datastructures, remain topics for further re-search. Data structures will always sit un-easily within pure dataflow models, butas their provision is virtually essential,it is an issue that must necessarily beexamined.

6.4. Nondeterminism

The deterministic nature of dataflowgraphs has been promoted many timesas an advantageous feature [Kahn1974; Karp and Miller 1966; Kosinski1973; Naggar et al. 1999; Verdoscia andVaccaro 1998]. This is because thedataflow concept lends itself well tomathematical analysis and proofs [Kahn1974] and nondeterminism would destroyor limit many essential properties. Weng[1975] observed that in von Neumannlanguages, concurrency constructs almostalways introduce unwanted concurrencyinto programs, and that developing dis-tributed systems in this model is madeextremely difficult by this fact. Mostdataflow programming languages aredeterminate, and the nondeterminacy insome of those that are not is not alwaysintentional—often being the result ofimperfect implementation decisions.Valid, Cajole, and DL1 have very limitednondeterminate features.

However, it is also widely accepted thatthere are many applications that actuallyrequire non-determinacy. These are sys-tems that are essentially operating in non-determinant environments, such as book-ing systems and database access systems.This was recognized early in the develop-ment of dataflow languages. Dennis [1974]conceded the point, and Kahn [1974], afterhis detailed mathematical analysis of de-terminate dataflow graphs, conceded that

his model was severely limited because itcould produce only determinate programs.While not dismissing the possibility of ex-tending the theory to nondeterminate pro-grams, the task appears daunting: Kahn[1974] remarked only that he did notthink it was impossible, but did not findit obvious how to do it satisfactorily. Thisview was supported by Kosinski [1978],who reported that attempts to formal-ize nondeterminate dataflow graphs hadbeen rather unsatisfactory due to theircomplexity.

The dichotomy in regard to nondeter-minism appears to be the result of a di-vision between those who wish to usedataflow as a means to ease the formalproof and analysis of programs and thosewho wish to use dataflow for all varietiesof programming problems. The former re-gard determinacy as essential, whereasthe latter regard the provision of nonde-terminacy as essential.

This problem can be resolved by pro-viding well-structured nondeterminacy. Inadmitting the need for nondeterminacy,Dennis [1974] nevertheless insisted thathe wanted to be able to guarantee users ofhis language that his program was deter-minate if they desired such a guarantee.Arvind et al. [1977] proposed that nonde-terminacy be permitted only by very ex-plicit means, to provide it for those whowant it, but guarantee its absence, if not.They demonstrated two constructs: thedataflow monitor and the nondeterminis-tic merge as vehicles for this.

The nondeterminate merge appears tobe able to solve many of the problems as-sociated with the lack of nondeterminacy.Semantically, it is a node that takes twoinput arcs and one output arc and mergesthe two streams in a completely arbitraryway. In most cases, such as a bookingsystem, this is all the nondeterminacy thatis required. The advantage of this is thatthe nondeterminism can be readily identi-fied. It is even possible to have nondeter-minate subgraphs within a graph that isotherwise determinate. Therefore, it maybe possible to apply mathematical princi-ples to the graph even if it does have non-determinate sections.



If dataflow is to become an accept-able basis for general-usage programminglanguages, nondeterminacy is essential.As well as having disadvantages for for-mal proofs, nondeterminacy also damagesthe software engineering process by mak-ing debugging more difficult. Therefore,the question is how to successfully con-trol the propagation of nondeterminacy indataflow systems, but still permit the soft-ware engineer to write usable programs.

7. CONCLUSION

In this article, the history of dataflow pro-gramming has been charted to the presentday. Beginning with the theoretical foun-dations of dataflow, the design and imple-mentation of fine-grained dataflow hard-ware architectures have been explored.The growing requirement for dataflow pro-gramming languages was addressed bythe creation of a functional paradigm oflanguages, and the most relevant of thesehave been discussed.

The discovery that fine-graineddataflow had inherent inefficiencies ledto a period of decline in dataflow researchin the 1980s and early 1990s. However,research in the field resumed in the 1990swith the acceptance that the best dataflowhardware techniques would come frommerging dataflow and von Neumanntechniques. This led to the developmentof hybrid architectures, whose primarytrait was a move away from fine-grainedparallelism toward more coarse-grainedexecution.

The most important development indataflow programming languages in the1990s was the advent of dataflow visualprogramming languages, DFVPLs, whichhave been explored. Integral to a DFVPLis its development environment, and thesehave been discussed. The change in moti-vation for pursuing DFVPLs toward soft-ware engineering has been noted. Finally,many issues remain open in dataflow pro-gramming, and four of these have beendiscussed.

Five key conclusions can be drawn re-garding the current state of dataflowprogramming.

—The major change in dataflow re-search as a whole has been the moveaway from fine-grained parallelism to-wards medium- and coarse-grainedparallelism.

—The major change in the past decadein dataflow programming has been theadvent of dataflow visual programminglanguages.

—As it is visualization that is key to avisual programming language, the dis-tinction between a dataflow visual pro-gramming language and its environ-ment has become blurred and the twomust now be treated as one unit.

—Dataflow languages increasingly de-serve to be treated as coordination lan-guages, an important area of researchwith the advent of heterogeneous dis-tributed systems.

—The three key open issues in dataflowprogramming remain the representa-tion of control-flow structures, the rep-resentation of data structures, and thevisualization of execution.

REFERENCES

ACKERMAN, W. 1982. Data flow languages. IEEEComput. 15, 2, 15–25.

ACKERMAN, W. B. AND DENNIS, J. B. 1979. VAL—A value-oriented algorithmic language: Prelim-inary reference manual. Tech Rep. 218. MIT,Cambridge, MA.

AMAMIYA, M., HASEGAWA, R., AND ONO, S. 1984.Valid, a high-level functional programming lan-guage for data flow machines. Rev. Electric.Comm. Lab. 32, 5, 793–802.

ARVIND AND CULLER, D. E. 1983. The tagged to-ken dataflow architecture (preliminary version).Tech. Rep. Laboratory for Computer Science,MIT, Cambridge, MA.

ARVIND AND CULLER, D. E. 1986. Dataflow architec-tures. Ann. Rev. Comput. Sci. 1, 225–253.

ARVIND, CULLER, D. E., AND MAA, G. K. 1988. As-sessing the benefits of fine-grain parallelism indataflow programs. Int. J. Supercomput. Appl. 2,3, 10–36.

ARVIND AND GOSTELOW, K. P. 1977. Some rela-tionships between asynchronous interpretersof a dataflow language. In Proceedings of theIFIP WG2.2 Conference on the Formal Descrip-tion of Programming Lanugages (St. Andrews,Canada).

ARVIND, GOSTELOW, K. P., AND PLOUFFE, W. 1977.Indeterminancy, monitors, and dataflow. In


32 Johnston et al.

Proceedings of the Sixth Symposium on Operat-ing System Principles, 159–169.

ARVIND, GOSTELOW, K. P., AND PLOUFFE, W. 1978. Anasynchronous programming language and com-puting machine, Tech. Rep. TR 114a. Universityof California, Irvine, Irvine, CA.

ARVIND, NIKHIL, R. S., AND PINGALI, K. K. 1989.I-structures: Data structures for parallel com-puting. ACM Trans. Program. Lang. Syst. 11, 4,598–632.

ARVIND AND NIKHIL, R. S. 1990. Executing a pro-gram on the MIT tagged-token dataflow archi-tecture. IEEE Trans. Comput. 39, 3, 300–318.

ARVIND AND THOMAS, R. 1980. I-structures: An effi-cient data type for functional languages. Tech.Memo 178. Laboratory for Computer Science,MIT, Cambridge, MA.

ASHCROFT, E. A. AND WADGE, W. W. 1977. Lucid, anonprocedural language with iteration. Comm.ACM 20, 7 (July), 519–526.

ASHCROFT, E. A. AND WADGE, W. W. 1980. Somecommon misconceptions about Lucid. ACMSIGPLAN Not. 15, 10, 15–26.

AUGUSTON, M. AND DELGADO, A. 1997. Iterative con-structs in the visual data flow language. Proceed-ings of the IEEE Conference on Visual Languages(VL’97, Capri, Italy). 152–159.

BACKUS, J. 1978. Can programming be liberatedfrom the von Neumann style? A functional styleand its algebra of programs. Comm. ACM 21, 8(Aug.), 613–641.

BARAHONA, P. AND GURD, J. R. 1985, Simulated per-formance of the Manchester multi-ring dataflowmachine. In Proceedings of the 2nd ICPC (Sept.).419–424.

BAROTH, E. AND HARTSOUGH, C. 1995. Visual pro-gramming in the real world. In Visual Object-Oriented Programming: Concepts and Environ-ments. Prentice-Hall, Upper Saddle River, NJ,21–42.

BERNINI, M. AND MOSCONI, M. 1994. VIPERS: Adata flow visual programming environmentbased on the Tcl language. In Proceedings of theWorkshop on Advanced Visual Interfaces. 243–245.

BHATTACHARYYA, S. S. 1996. Software Synthesisfrom Dataflow Graphs. Kluwer Academic Pub-lishers, Dordrecht, The Netherlands.

BIC, L. 1990. A process-oriented model for efficientexecution of dataflow programs. J. Parallel Dis-trib. Comput. 8, 42–51.

BIC, L., ROY, J. M. A., AND NAGEL, M. 1995. Exploit-ing iteration-level parallelism in dataflow pro-grams. In Advanced Topics in Dataflow Comput-ing and Multithreading. IEEE Computer SocietyPress, Los Alamitos, CA, 167–186.

BOHM, W., NAJJAR, W. A., SHANKAR, B., AND ROH, L.1993. An evaluation of coarse grain dataflowcode generation stretegies. In Proceedings of theWorking Conference on Massively Parallel Pro-gramming Models (Berlin, Germany).

BUCK, J. AND LEE, E. A. 1995. The token flow model.In Advanced Topics in Dataflow Computing andMultithreading. IEEE Computer Society Press,Los Alamitos, CA, 267–290.

BURNETT, M. M., BAKER, M. J. ET AL. 1995. Scalingup visual programming languages. IEEE Com-put. 28, 3, 45–54.

COMTE, D., DURRIEU, G., GELLY, O., PLAS, A., AND SYRE,J. C. 1978. Parallelism, control and synchro-nisation expression in a single assignment lan-guage. ACM SIGPLAN Not. 13, 1, 25–33.

COX, P., GILES, F., AND PIETRZYKOWSKI, T. 1989. Pro-graph: A step towards liberating programmingfrom textual conditioning. In Procedings of theIEEE Workshop on Visual Languages. 150–156.

COX, P. AND SMEDLEY, T. 1996. A visual languagefor the design of structured graphical objects. InProceedings of the IEEE Symposium on VisualLanguages. 296–303.

CULLER, D. E., GOLDSTEIN, S. C., SCHAUSER, K. E.,AND VON EICKER, T. 1995. Empirical study ofa dataflow language on the CM-5. In Ad-vanced Topics in Dataflow Computing and Mul-tithreading. IEEE Computer Society Press, LosAlamitos, CA, 187–210.

DAVIS, A. L. 1974. Data driven nets—A class ofmaximally parallel, output-functional programschemata. Tech. Rep. IRC Report. Burroughs,San Diego, CA.

DAVIS, A. L. 1978. The architecture and systemmethod of DDM1: A recursively structureddata driven machine. In Proceedings of the 5thAnnual Symposion on Computer Architecture(New York). 210–215.

DAVIS, A. L. 1979. DDN’s—a low level programschema for fully distributed systems. In Proceed-ings of the 1st European Conference on Paralleland Distributed Systems (Toulouse, France). 1–7.

DAVIS, A. L. AND KELLER, R. M. 1982. Data flow pro-gram graphs. IEEE Comput. 15, 2, 26–41.

DAVIS, A. L. AND LOWDER, S. A. 1981. A Samplemanagement application program in a graphi-cal data-driven programming language. In Di-gest of Papers Compcon Spring, February 1981.162–165.

DE JONG, M. D., AND HANKIN, C. L. 1982. Struc-tured data-flow programming. ACM SIGPLANNot. 17, 8, 18–27.

DENNIS, J. 1977. A Language Design for Struc-tured Concurrency. The Design and Implementa-tion of Programming Languages. Lecture Notesin Computer Science, vol. 54. Springer-Verlag,Berlin, Germany, 23–42.

DENNIS, J. B. 1974. First version of a data flow pro-cedure language. In Proceedings of the Sympo-sium on Programming (Institut de Programma-tion, University of Paris, Paris, France). 241–271.

DENNIS, J. B. 1980. Data flow supercomputers.IEEE Comput. 13, 11 (Nov.), 48–56.



DENNIS, J. B. AND MISUNAS, D. P. 1975. A prelimi-nary architecture for a basic data-flow processor.In Proceedings of the Second Annual Symposiumon Computer Architecture. 126–132.

GAJSKI, D. D., PADUA, D. A., KUCLE, D. J., AND KUH, R. H.1982. A second opinion on data-flow machinesand languages. IEEE Comput. 15, 2, 58–69.

GAO, G. R. AND PARASKEVAS, Z. 1989. Compiling fordataflow software pipelining. In Proceedings ofthe Second Workshop on Languages and Com-pilers for Parallel Computing.

GELERNTER, D. AND CARRIERO, N. 1992. Coordina-tion languages and their significance. Comm.ACM 35, 2, 97–107.

GELLY, O. 1976. LAU software system: A high-leveldata-driven language for parallel processing. InProceedings of the International Conference onParallel Processing (New York, NY).

GHITTORI, E., MOSCONI, M., AND PORTA, M. 1998.Designing and testing new programming con-structs in a data flow VL. Tech. Rep. Universitadi Pavia, Pavia, Italy.

GLAUERT, J. R. W. 1978. A single assignment lan-guage for data flow computing, Master’s thesis.University of Manchester, Manchester, U.K.

GREEN, T. R. G. AND PETRE, M. 1996. Usabilityanalysis of visual programming environments:A “Cognitive Dimensions” Framework. J. Vis.Lang. Comput. 7, 131–174.

GURD, J. R. AND BOHM, W. 1987. Implicit parallelprocessing: SISAL on the Manchester dataflowcomputer. In Proceedings of the IBM-Europe In-stitute on Parallel Processing (Aug., Oberlech,Austria).

HANKIN, C. L. AND GLASER, H. W. 1981. The data-flow programming language CAJOLE—an infor-mal introduction. ACM SIGPLAN Not. 16, 7, 35–44.

HARVEY, N. AND MORRIS, J. 1993. NL: A general pur-pose visual dataflow language, Tech. Rep. Uni-versity of Tasmania, Tasmania, Australia.

HARVEY, N. AND MORRIS, J. 1996. NL: A parallel pro-gramming visual language. Australian Comput.J. 28, 1, 2–12.

HELSEL, R. 1994. Cutting Your Test DevelopmentTime with HP VEE. Prentice-Hall, HP Profes-sional Books, Englewood Cliffs, NJ.

HILS, D. D. 1992. Visual languages and comput-ing survey: Data flow visual programming lan-guages. J. Vis. Lang. Comput. 3, 1, 69–101.

HURSON, A. R., LEE, B., AND SHIRAZI, B. 1989. Hy-brid structures: A scheme for handling datastructures in a data flow environment. In Pro-ceedings of the Conference on Parallel Architec-tures and Languages (PARLE), 323–340.

IANNUCCI, R. A. 1988. Towards a dataflow/vonNeumann hybrid architecture. In Proceedings ofthe ICSA-15 (Honolulu, HI). 131–140.

IWATA, M. AND TERADA, H. 1995. Multilateral di-agrammatical specification environment basedon data-driven paradigm. In Advanced Topics in

Dataflow Computing and Multithreading. IEEEComputer Society Press, Los Alamitos, CA, 103–112.

JAGANNATHAN, R. 1995. Coarse-Grain DataflowProgramming of Conventional Parallel Comput-ers. In Advanced Topics in Dataflow Computingand Multithreading. IEEE Computer SocietyPress, Los Alamitos, CA, 113–129.

KAHN, G. 1974. The semantics of a simple lan-guage for parallel programming. In Proceed-ings of the IFIP Congress 74 (Amsterdam, TheNetherlands). 471–475.

KARP, R. AND MILLER, R. 1966. Properties of a modelfor parallel computations: Determinacy, termi-nation, queueing. SIAM J. 14, 1390–1411.

KELLER, R. M. 1985. Rediflow architectureprospectus, Tech. Rep. No. UUCS-85-105. De-partment of Computer Science, University ofUtah, Salt Lake City, Utah.

KELLER, R. M. AND YEN, W. C. J. 1981. A graphicalapproach to software development using func-tion graphs. In Digest of Papers Compcon Spring,February 1981. 156–161.

KIMURA, T. AND MCLAIN, P. 1986. Show and Telluser’s manual. Tech. Rep. WUCS-86-4. Depart-ment of Computer Science, Washington Univer-sity, St Louis, MO.

KIPER, J., HOWARD, E., AND AMES, C. 1997. Criteriafor evaluation of visual programming languages.J. Vis. Lang. Comput. 8, 2, 175–192.

KOSINSKI, P. 1978. A straightforward denotationalsemantics for non-determinate data flow pro-grams. In Proceedings of the 5th ACM Sympo-sium on Principles on Programming Languages.ACM Press New York, NY.

KOSINSKI, P. R. 1973. A data flow language for op-erating systems programming. In ‘Proceedingsof ACM SIGPLAN-SIGOPS Interface Meeting.’SIGPLAN Not. 8, 9, 89–94.

LABVIEW. 2000. Lab View User Manual. NationalInstruments, Austin, TX.

LEE, B. AND HURSON, A. R. 1993. Issues in dataflowcomputing. Adv. in Comput. 37, 285–333.

LEE, B. AND HURSON, A. R. 1994. Dataflow architec-tures and multithreading. IEEE Comput. 27, 8(Aug.), 27–39.

LEE, E. 1997. A denotational semantics fordataflow with firing. Memorandum UCB/ERLM97/3. Electronics Research Laboratory, Uni-versity of California, Berkeley, Berkeley, CA.

LEE, E. AND MESSERSCHMITT, D. 1987. Staticscheduling of synchronous dataflow programsfor digital signal processing. IEEE Trans.Comput. C-36, 1, 24–35.

LEE, E. AND PARKS, T. 1995. Data-flow process net-works. Proc. IEEE. 83, 5, 773–799.

MCGRAW, J., SKEDZIELEWSKI, S. ET AL. 1983.SISAL—Streams and Iteration in a SingleAssignment Language Reference Manual (Ver-sion 1.0). Livermore National Laboratory,Livermore, CA.


34 Johnston et al.

MORRISON, J. P. 1994. Flow-Based Programming:A New Approach to Application Development.van Nostrand Reinhold, New York, NY.

MOSCONI, M. AND PORTA, M. 2000. Iteration con-structs in data-flow visual programming lan-guages. Comput. Lang. 26, 2-4, 67–104.

NAGGAR, W., LEE, E., AND GAO, G. R. 1999. Ad-vances in the dataflow computational model.Parallel Comput. 25, 1907–1929.

NAJJAR, W. A., ROH, L., AND WIM, A. 1994. An evalu-ation of medium-grain dataflow code. Int. J. Par-allel Program. 22, 3, 209–242.

NIKHIL, R. S. AND ARVIND 1989. Can dataflow sub-sume von Neumann computing? In Proceedingsof the ICSA-16 (Jerusalem, Israel). 262–272.

NING, Q. AND GAO, G. R. 1991. Loop storage opti-mization for Dataflow machines. ACAPS Tech.Memo 23. School of Computer Science, McGillUniversity, Montreal, P. Q., Canada.

PAPADOPOULOS, G. M. 1988. Implementation of ageneral purpose dataflow multiprocessor. Tech.Rep. TR432. Laboratory for Computer Science,MIT, Cambridge, MA.

PAPADOPOULOS, G. M. AND TRAUB, K. R. 1991. Multi-threading: A revisionist view of dataflow archi-tectures. MIT Memo CSG-330. MIT, Cambridge,MA.

PLAICE, J. 1991. RLUCID, A General Real-TimeData-Flow Language. Lecture Notes in Com-puter Science, vol. 571. Springer-Verlag, Berlin,Germany, 363–374.

RASURE, J. AND WILLIAMS, C. 1991. An integrateddata flow visual language and software devel-opment environment. J. Vis. Lang. Comput. 2,217–246.

RICHARDSON, C. 1981. Manipulator control using adata-flow machine. Doctoral dissertation. Uni-versity of Manchester, Manchester, U.K.

SAKAI, S., YAMAGUCHI, Y., HIRAKI, K., KODAMA, Y., AND

YUBA, T. 1989. An architecture of a dataflowsingle chip processor. In Proceedings of the 16thInternational Symposium on Computer Architec-ture. 46–53.

SEROT, J., QUENOT, G., AND ZAVIDOVIQUE, B. 1995. Avisual dataflow programming environment for areal time parallel vision machine. J. Vis. Lang.Comput. 6, 4, 327–347.

SHIZUKI, B., TOYODA, M., SHIBAYAMA, E., AND TAKAHASHI,S. 2000. Smart browsing among multiple as-pects of data-flow visual program execution,using visual patterns and multi-focus fisheyeviews. J. Vis. Lang. Comput. 11, 5, 529–548.

SHURR, A. 1997. BDL—a nondeterministic dataflow programming language with backtracking.In Proceedings of the IEEE Conference on VisualLanguages (VL’97, Capri, Italy).

SILC, J., ROBIC, B., AND UNGERER, T. 1998. Asyn-chrony in parallel computing: from dataflow tomultithreading. Parallel Distrib. Comput. Pract.1, 1, 3–30.

STERLING, T., KUEHN, J., THISTLE, M., AND ANASTASIS,T. 1995. Studies on Optimal Task Granular-ity and Random Mapping. In Advanced Topics inDataflow Computing and Multithreading. IEEEComputer Society Press, Los Alamitos, CA, 349–365.

TANIMOTO, S. 1990. VIVA: A visual language forimage processing. J. Vis. Lang. Comput. 1, 127–139.

TOYODA, M., SHIZUKI, B., TAKAHASHI, S., MATSUOKA,S., AND SHIBAYAMA, E. 1997. Supporting designpatterns in a visual parallel data-flow program-ming environment. In Proceedings of the IEEESymposion on Visual Languages (Capri, Italy).76–83.

TRELEAVEN, P. C., BROWNBRIDGE, D. R., AND HOPKINS,R. P. 1982. Data-driven and demand-drivencomputer architecture. ACM Comput. Surv. 14,1, 93–143.

TRELEAVEN, P. C. AND LIMA, I. G. 1984. Future com-puters: Logic, data flow, . . . , control flow? IEEEComput. 17, 3 (Mar.), 47–58.

VEEN, A. H. 1986. Data flow machine architecture.ACM Comput. Surv. 18, 4, 365–396.

VERDOSCIA, L. AND VACCARO, R. 1998. A high-leveldataflow system. Comput. J. 60, 4, 285–305.

WADGE, W. W. AND ASHCROFT, E. A. 1985. Lucid, theDataflow Programming Language. APIC Stud-ies in Data Processing, no. 22. Academic Press,New York, NY.

WAIL, S. F. AND ABRAMSON, D. 1995. Can DataflowMachines be Programmed with an ImperativeLanguage? In Advanced Topics in DataflowComputing and Multithreading. IEEE Com-puter Society Press, Los Alamitos, CA, 229–265.

WATSON, I. AND GURD, J. R. 1979. A prototype dataflow computer with token labelling. In Proceed-ings of the National Computer Conference. 623–628.

WENG, K. S. 1975. Stream oriented computation inrecursive data-flow schemas. Tech. Rep. 68. Lab-oratory for Computer Science, MIT, Cambridge,MA.

WHITING, P. AND PASCOE, R. 1994. A history of data-flow languages. IEEE Ann. Hist. Comput. 16, 4,38–59.

WHITLEY, K. 1997. Visual programming languagesand the empirical evidence for and against.J. Vis. Lang. Comput. 8, 1, 109–142.

YU, Y. J. AND D’HOLLANDER, E. H. 2001. Loop paral-lelization using the 3D iteration space visualizer.J. Vis. Lang. Comput. 12, 2, 163–181.

Received October 2001; revised September 2002, September 2003; accepted April 2004


Advances in Dataﬂow Programming Languages - … · Advances in Dataﬂow Programming Languages...

Documents

Transcript of Advances in Dataﬂow Programming Languages - … · Advances in Dataﬂow Programming Languages...