DEMOB - An Object Oriented Application Generator for Image Processing

N. Bryson, D.H.Cooper, J.G.Graham, D.P.Pycock, C.J.Taylor, P.W.Woods

Wolf son Image Analysis Unit, Department of Medical Biophysics

The Victoria University of Manchester, Manchester M13 9PT

This paper describes an Object Oriented programgenerator for image processing applications. Controlis represented by a dataflow graph and interaction by"view" objects which update displays and modifydomain objects. Progress so far also indicates thatObject Oriented Programming for user-defined imageprocessing requires a rich programming supportenvironment.

Many groups have developed libraries of imageprocessing (IP) modules for applications in themedical and industrial fields. The use of theselibraries requires considerable programming effortand expert knowledge, which limits the economicviability of the technology. To develop an application,the user must not only have expert knowledge ofimage processing algorithms, but must also contendwith the messy details of programming. Because ofthe commercial pressure to quickly demonstrate thefeasibility of program designs, there is a need for anapplication generator. This paper describes ourattempts to develop such a tool (DEMOB). Theprimary aim of this tool is not to help with the "art" ofimage processing, but to help make the constructionof an application as direct as possible.

Objectives of DEMOB

One of the objectives in designing DEMOB was toinvestigate the problems involved in designing anapplication generator. Prototyping tools implementedusing conventional structured programming sufferfrom having the structure of domain objects,interactions, and control programmed in. Forexample the implicit control structure may be apipeline, along which a work image flows, with theuser being offered (via menus) a choice of theoperations to be performed on the image.Interactions occur in a pre-programmed manner,

This work is supported by an SERC grant and ispart of ALVEYproject MMI-093: "Techniques forUser Programmable Image Processing (TUPIP)"

and tedious re-coding is required when newoperations are added.

Our design goals included the requirement thatcontrol be separated out explicitly from the operationof the tool and from the domain objects. Interactionsshould also be factored out, so that interactive toolsfor different domains could be rapidly built from a setof building blocks. The program generator should beopen, so that further domain objects and interactionscould be added easily. The user should not beconstrained to follow a particular design cycle, butinstead should be able to interact withsubcomponents of the problem in a relativelyunstructured way. The Object Oriented Programming(OOP) technique1, with its desirable properties ofinformation hiding and run-time binding seemed tooffer a viable approach to implementing such a tool.One of the aims of the project was to explore theuses of OOP for IP. Because of the nature of IP, withthe need to display and interact with raw andprocessed images, the tool should have a mouse -driven, window based graphical interface. A briefdescription of the OOP paradigm is given below. Fora fuller description, the reader is referred toreference 2.

APPLICATION REPRESENTATION

Separation of Control

A prototype application is represented internally by adataflow graph3. This representation was chosenbecause the explicit recording of dependenciesbetween data items provides the potential forautomated reasoning about applications. A simplegraph, representing "blob" extraction from animage, is shown in figure 1. The dataflow graphconsists of: nodes, representing actions applied todata objects; tokens, representing references to dataobjects; and directed arcs, which transfer referencesto tokens between nodes.

A node is ready to fire when it has received all itstokens from its input arcs. New tokens are created bythe node, and are transferred to further nodes via the

37

AVC 1988 doi:10.5244/C.2.7

output arcs, making them ready to fire. Afterexecution only the "dangling" output arcs of thegraph contain references to tokens, all intermediatedata objects (and their tokens) having beenconsumed. Each token contains a counter forreferences to the data object. When references totokens are created or consumed, the counter isincremented or decremented, respectively. Whenthe counter reaches zero, the data object, andtoken, can be deleted.

1 Define datal I Define datal I Define datal^Maaaa^^^B—^nMaaHHaM ^ ^ ^ ^ ^ | ^ ^ ^ ^ ^ ^^^m^^m^m^m^^m^^^^m^^

I Define Data!fValue_range 1

List of blobs

Figure 1. Graph representing blob extraction.

new nodes, and adding them to the graph. Graphsand objects can also be saved and retrieved fromdisc file.

IMPLEMENTATION

The Object Oriented ProgrammingEnvironment

The key features of OOP are data encapsulation andinheritance. The OOP technique consists ofidentifying objects which are to be manipulated in anapplication, and in defining the data structures andoperations needed for each object. The privatedata structure representing the object is protectedfrom direct manipulation by the user, and operationson it may only be carried out by sending a messageto the object, which uses it to select an appropriateoperation. It is important to note that the messageonly conveys "what" the programmer wants done,but the object itself decides "how" it is done. Forexample the message AREA sent to a shape objectmay cause different operations to be carried out,depending on whether the shape is a circle,rectangle, etc. Because the internal representation ishidden, it can be changed without affecting the user,who need only know the messages to which theobject responds.

The User Model

DEMOB is designed to present the application anddata objects graphically to the user. The user mayselect any action or data object in his application byselecting the appropriate node or arc of the graph,which will respond by offering an interacton with theappropriate object. Each interaction takes placewithin a window, which contains a menu, a display ofthe object, and a prompt line. Selection within thewindow leads to a more specialised interaction withthe object, or an interaction with a new object. Initiallythe user is offered a general interaction with anempty list object. Typically the user then chooses toload a graph from a disc file, or to create a newexample. An interaction with the new graph is enteredand the user is presented with a diagram such as thatshown in figure 1, and a menu of availableinteractions. For example, if the user selects adangling output arc he will enter an interaction withthe data object on the arc. The arc itself can beselected, and modified so that, during execution, itwill display the data objects passing through it.

During execution the arcs and nodes are highlightedas they become active, and windows open on thescreen to show data objects as they are created. Theuser modifies his program by cutting arcs, creating

Heap(Objects)

Static data segment(Classes)

Text Segment(Methods)

Superclass : NullName : Object

Class variables

Class method table

NEW : newObjeotQ

Instance method table

PRINT : printObjectQ

Name : TomClass : Cat Q

Instancevariables

Superclass : ObjectName : Mammal

Class variables

Class method table >

NEW : newMammal

Instance method table3RINT : printMammal

0

Superclass: MammalName : Cat

Class variables

Class method table

NEW : newCat

Instance method tabl<

PRINT

i I

Figure 2. Schematic memory map of OOPsystem

An individual object is regarded as an instance of aparticular class e.g. Tom is an instance of the class

38

Cat. The class definition defines the data structure("instance variables") of its instances and themessages (with corresponding operations or"methods") to which it will respond, in the form of alookup table, indexed by the message selectors.Each object contains a pointer to its class, which isitself an object, with corresponding class methodsand "class variables". In particular, each class canrespond to the message NEW by returning a newinstance (by invoking the newCat method). Figure 2shows a schematic memory map showing therelationship between objects, classes and methods.

The use of encapsulation helps to protect code fromthe effects of changes. The use of inheritance allowscode to be re-used. The classes are arranged in aninheritance hierarchy, with more general operationsand instance variables being defined in classes highin the hierarchy. For simplicity, each class caninherit from one "superclass" only, although in otherimplementations, multiple inheritance is a usefulfeature. Each class contains a pointer to itssuperclass (except for Object, the class at the root ofthe hierarchy). Methods high in the hierarchy mayonly manipulate instance variables defined at thatlevel or higher in the hierarchy.

We have developed an OOP environment in-house inwhich to implement DEMOB. This consists of C withsome preprocessor tools. The programmer causesmessages to be sent to objects by inserting code ofthe form:-

Tom = NEW$(Cat);PRINT$(Tom);

where Tom is of type "ob_ptr" i.e. a pointer to anobject. A special global object pointer, "self",represents the receiver of the current message, andis used within methods to access the instancevariables of the object. A preprocessor tool checksthat the message selector is represented in a file ofvalid messages, and then converts this string into acall to a message handling routine:

Tom = msg(NEW.Cat);msg(PRINT.Tom);

The message handling routine expects the messageselector and the receiver of the message as the firsttwo parameters. Upon execution, the messagehandling routine searches the method lookup tableof the class of the object for the correspondingmethod (it recognises a class by the fact that itsclass pointer instance variable is null). The stack isadjusted to simulate a standard C procedure call, andcontrol is passed to the method.

The programmer may also specify that the search fora method should begin in the superclass, rather thanin the class, by writing:-

PRINT$super(Tom);

This powerful facility allows the chaining together ofmethods. For example, the PRINT message causesthe printing of the values of all instance variables ofthe object, and is implemented at each level of theclass hierarchy by methods of the form:-

PRINT$super(self);printf("Instance variables at this level");

This causes the instance variables, starting at theroot of the hierarchy, to be printed.

The low level image processing routines of oursystem are microcoded to make use of particulardata structures, optimised for IP operations andexecuted on a slave co-processor. Since a largeeffort had already been expended in developingexisting software and hardware,we were constrainedto include these existing data structures in DEMOB.One of the objectives in developing DEMOB was toinvestigate the effective use of mixed typed datastructures and objects.

Image Processing Objects

I Laplaclan]

Figure 3. Image Processing ObjectsClass Hierarchy

Since we wish to be able to add new classes andmodify existing classes without too much disturbanceto the rest of the system, DEMOB makes noassumptions about the domain classes, other thanthat they will respond to a limited set of messages,which provide the interface between these classesand DEMOB. These messages allow DEMOB toascertain the default method of interaction and theoperations available for the domain object. Besidesgeneral purpose objects such as integers, reals,booleans, lists, etc., a variety of data objects specificto IP are also required. These include images,cameras, pointsets, value ranges, and imagetransform specifiers4. A class hierarchy of these

39

objects is shown in figure 3.Associated with each object is a set of messageswhich are used to process the data e.g. a Cameraobject responds to the PHOTO message by producingan Image object. The processing steps in theapplication are built up using these messages, whichare inserted into the graph as described below. Alsoassociated with each object is a set of interactionse.g. the Real number object has a set of interactionsallowing the number to be modified in a variety ofways. These interactions can be inserted into thegraph as described below.

Graph Nodes

The structure of the dataflow graph allows controlover the order in which operations are carried out.Further control is provided by supplying specialisednodes which implement further control structures.The class hierarchy of the objects used inconstructing graphs is shown in figure 4. The Nodeclass provides the functionality common to all nodese.g. the ability to receive and transmit tokens.Further, more specialised functionality is provided bythe subclasses of Node.

Figure 4 . Inheritance hierarchy ofgraph objects

One such class is the "Data Definition" class, whoseinstances act as sources of data for the dataflowgraph. When the node is created the user is asked todefine the data object, and on subsequent executiona token representing the data object is output by thesource node. In the example graph shown in figure 1,four data objects are needed : a camera, a maskimage, a pointset (defining the region over which theoperation is done) and a value range (which definesthe thresholds for the slice operation).

Multiple references to data objects are frequentlyrequired, so a "copy" node class is provided. Thesework by incrementing the reference counter of theinput token, and passing out multiple references to itvia the output arcs.

A "graph" node, like any other, can receive tokensfrom its input arcs, and passes tokens to its outputarcs. Internally its execution is represented by acollection of nodes and arcs i.e. a sub-graph. Agraph can thus have a layered structure, with a"root" graph node controlling the execution of itssub-graph, which itself can contain graph nodes.

Iterative execution is implemented by providing avariety of "iteration" node classes. For example, aspecialised type of graph node, the "while-do" nodecontrols the execution of a condition graph and anaction graph. During execution the set of input tokensis passed to the condition graph, which executes andreturns a boolean object. If it is "true" the set of inputtokens is passed to the action graph, which returns anew set of tokens, and the cycle is repeated. If theboolean is "false", the tokens are passed out asoutput.

Conditional execution is implemented by providing an"if-then-else" node class. This node controls acondition graph, a "true" graph and a "false" graph.The execution cycle is similar to that of the"while-do" graph i.e. the input tokens are passed tothe condition graph, and a boolean object isreturned. Depending on its value, the input tokensare passed to the "true" or "false" graph.

These nodes introduced so far provide the controlstructures of conventional structured programming (asubroutining facility could easily be added). A furtherclass of node, the "personalised" node, is includedto allow user selected actions on data objects.Actions are carried out by sending a message to anobject e.g. to add two numbers together themessage is : -

C = ADD$(A,B);

where A, B,C are references to the number objects,ADD is the message, and A is the receiver of themessage. The personalised node has instancevariables allowing it to store this message. Theconvention adopted is that the message is sent to thedata object input via the first input arc, with the dataobjects from the other input arcs as parameters. Thereturned object from the message is assumed to bea list of output data objects, which are transferred tothe output arcs. In this case a single object, C, isreturned. During the creation of a personalised node,the user is prompted to define a receiver, which thenoffers for selection all the available messages for thatclass of object.

40

Part of the user's application may involve run-timeinteraction with the data objects, and a special"interaction node" class is provided to allow this.When creating this node, the user will be prompted todefine the class of input, and will be asked to selectfrom all available interactions with this class. Aninstance of this interaction (see below) will becreated and placed in the interaction node. When thedata object arrives along the input arc, the interactionwill take place, and the modified data object will beoutput along the output arc.

The Interaction Model

The programming of interaction required a large partof the effort in developing DEMOB. A graphicssub-system was developed, which implements aGKS5 model of graphics i.e. overlapping rectangularregions of the display device, called "viewports",display "icons" in "worlds" through "windows"opening on the worlds. Viewports are groupedtogether in frames. These components wereimplemented by defining classes of objects, with alldevice-dependent code being confined to the Screenclass.

Class 2

| Default 1 |

•

" \ ~ ~ ~ - —

Interaction 2

SkelV 2

I Default 2 1 Interaction 3

Figure 5. The View Class Hierarchy

Rather than have each domain object controlling itsinteractions, we decided that each interaction shouldbe defined by a separate class, to increasemodularity and to save memory space. When aninteraction with an object is required, an instance ofthe relevant class, known as a "view", is created andis put in control of the object. The view object would

inherit the instance variables and most of themethods needed to represent and manage thedisplay from its superclasses, and need only providelocally the code needed to determine the course ofthis particular interaction.

We thus define a class hierarchy of views toaccompany the hierarchy of domain objects, asshown in figure 5 . This view hierarchy contains"skeleton" classes which provide all the instancevariables and methods common to interactions, and"view" classes which define particular interactions.Each object has a default view class, whichimplements the most general interaction possible i.e.to display the object and to offer all interactionsavailable for it. It also allows the user to selectsub-components of the object for further interaction.When a new domain class is added a correspondingskeleton class with default view and any further viewclasses are also added.

RESULTS

A basic version of DEMOB has been implemented ona CVAS 3000 (Visual Machines Ltd.) system. Imagesof the screen, showing a sample graph during andafter execution, are shown in figures 6 and 7. Theseshow a graph interaction window, with associatedmenu, and displays of a source image and a slicedimage.

Figure 6. Sample graph during execution

41

Figure 7 - Sample graph after execution

DISCUSSION

Limitations of the OOP Environment

Designing with objects proceeds by identifying theinstance variables and methods of classes. As thedesign proceeds, the programmer identifies ways inwhich code can be reused, usually by spotting waysof adding new classes and by splitting or movingmethods and instance variables around in the classhierarchy. The final system contains a large numberof classes, each with a relatively small number ofmethods, each containing a small number of lines ofcode. It is easy for the programmer to lose track ofsuch a system, and it can be difficult for a newprogrammer to add new functionality to the system,since in order to do so, he must understand what heis inheriting. We thus need tools to allow theprogrammer to browse around the source code in astructured manner such as are supplied in Smalltalk6.

In our OOP environment, all class and methoddefinitions are compiled. While this increasesrun-time efficiency, it also means that the definitionof an object is fixed. Further, although the code iscompiled, no static type-checking is done.

Design Issues Raised in OOP

Initially, design concentrated on using the"information hiding" aspect of OOP, but as wegained experience, inheritance gained in importance.The derivation of a class hierarchy can be difficult, inpart because the only relationships provided are the

"is-a" and "is-an-instance-of" relations. These donot, however, describe the relationship between say,views and their objects. For these objects the"information hiding" aspect of OOP is adisadvantage, since the views need to have accessto the instance variables of their objects. Thisproblem arises because it is difficult to decompose IPapplications into independent static classes ofobjects i.e. where information can be encapsulatedpermanently into objects. In practice we wish to beable to merge the information from several classes ofobject e.g. in applying an operation over a pointset inan image. Where a static encapsulation ofinformation suffices(e.g. the dataflow graph) themodel works well. The frame paradigm 7 presents aricher environment for the representation of domainknowledge,allowing object decomposition as well asmore complex relationships between objects.

Our experience has shown that the use of mixedtyped and object variables reduces the advantages ofusing OOP, since much code had to be handcraftedto deal with the typed variables. Of course, these areneeded to map onto the IP software and hardware.The correct way to deal with them is to implement abasic set as objects, with methods handcoded toimplement the functionality provided automatically forother objects. It is instructive to compare ourapproach with the Eiffel OOP language 8 . Thislanguage has a compiler which recognises the use ofa limited set of simple types, but any complex datastructure must be represented as an object.

Design Issues in the User Interface

The user interface is crucial to the success of a toolsuch as DEMOB. Users are sensitive to featureswhich appear relatively trivial, such as the particularstyle of interactions, or the wording of prompts.There are a variety of ways in which an interaction,such as modifying an integer, can be implemented.One solution is to recognise that these interactionshave much in common, and thus could berepresented by an appropriate view class hierarchy.

Instead of being presented with the dataflow graph,some users felt that it would be better for this to behidden, and instead to present only the data objects.Design would proceed by creating and selectingobjects for processing. The user would then selectsan appropriate operation which could take theseobjects as inputs. The selected action would beinvoked, causing new objects to be created.

CONCLUSIONS

The dataflow graph is adequate for representing anapplication. However the need to fully specify thedataflow is tedious for programmers, since the

42

normal constructs of conventional programminglanguages correspond to a lot of dataflow, which theprogrammer does not normally need to specifyexplicitly.

While OOP greatly increases the robustness andreuseability of code, and provides a useful paradigmfor the development of complex software systems,the behaviour and relationships between the complexdata structures used in IP are not adequatelymodelled by message passing objects arranged in asingle inheritance class hierarchy. To use OOP thedevelopment environment should be fully integratedwith the language to allow rapid modification of anevolving system. A clean interface between typelessdata structures (objects) and typed data structuresshould be maintained. The dataflow graph provides aviable means of representing procedural knowledge,but needs careful design of the user interface inorder to gain user acceptability.

REFERENCES1. ACM SIGPLAN Notices, Vol. 21, No. 10, Oct.

19862. Brad C. Cox, "Object Oriented Programming -

An Evolutionary Approach", Addison - Wesley,1986

3. /£££ Computer, Special Issue on DataflowSystems, Vol. 15, No. 2, Feb. 1982

4. Graham J., Taylor C.J., Dixon R.N., "Acompact set of image processing primitivesand their role in a successful applicationprogram", Patt. Recog. Lett., ±, 325 - 333,

19875. Hopgood F.R.A., Duce D.A., Gallop

J.R., Sutcliffe D.C.," introduction to theGraphics Kernel System (GKS)" A.P.I.C.Studies in Data Processing No. 19,Academic Press, 1983

6. Goldberg A., Robson D., " Smalltalk - 80 TheLanguage and its Implementation", Addison -

Wesley Series in Computer Science, 19837. Wood P.W., Pycock D.P., Taylor C.J, "A

Frame-based System for Modelling andExecuting Visual Tasks", Alvey Conference1988.

8. Meyer B., "Eiffel: Programming for Reusabilityand Extendability", SIGPLAN NoticesVol. 22., No. 2, Feb. 1987

43

44