Design of a multifunctional communications software package

A modular multifunctional package designed to adapt easily to system changes is described by Michael Schneider and Leo May.

Data communications and computer networks have under- gone phenomenal growth, resulting in the profifemtion of hardware, software, and communications protocols. How- ever, as needs change and new technologies are developed existing software frequently becomes incapable of handling the new configurations and therefore obsolete. The design and implementation of a new data communications software package, Commpac, designed to address these problems, is described. The package is modular and multi- functional, and supports the hierarchy of point-to-point communications protocols. In addition, it has been designed to adapt to system changes and to provide the software resources necessary for the development o f new modules to cope with these changes.

Data communications and computer networks are areas cur- rently undergoing a phenomenal growth and change. Both manufacturers and suppliers are constantly introducing new device controllers, interfaces, terminals, protocols, and end- user applications. In addition, there is a great deal of international activity to standardize transmission media and communication protocols. This overall volatility has resulted in proliferation of communications hardware and software packages. Then, as needs change, or new pertur- bations enter the system, it is frequently found that the existing software is no longer capable of su pporting the new specifications and subsequently becomes obsolete. This continuous development of new communications software is expensive and leads to a duplication of effort. Worst of all, the new software often lacks the flexible design needed to survive the inevitable next round of change, thus perpet- uating the problem.

Because of this rapid rate of change, the development of any general purpose data communication package should concentrate on achieving two very specific and desirable characteristics:

Multifunctionality: it should address the entire range of tasks encountered within a communications environment. This would allow those parts of the package common to

Computer Science Department, 136 Lind Hall, Institute of Technol- ogy, University of Minnesota, Minneapolis, Minnesota 55455, USA

a number of different, but related, tasks to be shared thus avoiding the duplication of effort. Modularity: it should be composed of small independent units, subsystems, which can be arbitrarily changed without causing an unexpected change to any other- subsystem. The package may then be easily updated in response to new needs, new standards or new equipment.

MU L T I F U N C T I O N A L I T Y

The functions of a communications software package can be divided into two distinct classes: support and emulation.

Support means the interfacing of the communications package to both message source e.g. terminals, end-user and a message sink e.g. host computer in a true point-to point communications environment. The system described here does not address end-to-end transport-level protocols defined by the ISO Open System Interconnection Working Group TC97. The protocols for exchanging information within a network are typically hierarchical, so the support functions of a communications package are also best viewed in a hierarchical fashion 1. This point is illustrated in Figure 1.

A communications package must be able to support a wide range of device controllers with the enormous range of characteristics shown in Table 1. As new controllers enter the market the package must be easily modifiable to accom modate their new characteristics without affecting other levels of support. The package must be able to handle a wide range of datalink control (DLC) protocols which support the exchange of messages across a physical link. As new DLCs are developed, either by manufacturers - e.g. IBM SDLC,

Host computer

~ C ..... ications~ Line ~7 controller software ,

L Device control support Link control support

Process level support

Figure I. Hierarchical levels o f communications support

Transmission media

98 0140-3664/79/030098-07 $02.00 © 1979 IPC Business Press computer communications

Table 1. Typical support characteristics within each level of the communications hierarchy

A Device control support

Synchronous, asynchronous Programmable, non-programmable Parallel, bit-serial Word mode, DMA Single-line control, multiline control Parity, CRC-12, CRC-16 RS-232, current loop, digital Programmable code set, character size, speeds

B Link control support

Half-du plex, full-du plex Point-to-point, multipoint Bit oriented, character oriented Variability in message headers, trailers, lengths Variability in control message types

C Process control support

Process naming conventions Process activation/deactivation Operating system support services and macros

(e.g. file transfer, virtual terminal protocols) Buffering conventions Device dependent input/output services

Digital Equipment Corporation DDCMP - or by inter- national convention - e.g. ANSI ADDCP, ISO HDLC - the communications package must be able to be modified quickly and easily to accommodate the semantics of.the new protocol. Finally the package must support process level interactions, with an application process or end user, through an operating systems interface. As the operating system of a particular computer is changed and modified over time, the communications package running on, or interacting with that computer, should be able to quickly adapt to the new system. And, again, the necessary modifi- cations should be totally transparent to the lower level device and link control subsystems.

In addition to the three levels of support described above, there is a second class of communication functions which is frequently overlooked in the design of communications soft- ware packages - emulation. Emulation implies the creation and manipulation of a virtual, i.e. hypothetical, communica- tions environment in order to study the behaviour and per- formance characteristic of a proposed or potential configur- ation. It is primarily this emulation concept of Commpac that provides the multifunctionality and distinguishes the system from other communications packages in the market- place (e.g. ICM's CICS).

Emulation would be a useful tool in hardware diagnositcs. Failures in communication devices such as terminals could be effectively investigated by emulating and studying the conditions which caused the failure. Service routines, which are designed as an integral part of the communications

package, could collect, analyse, and present the data result- ing from the emulation, significantly reducing the time and effort needed to diagnose the problem. Software diagnos- tics could also be handled through an emulation mode. When communication protocols, either at the link or process level, are updated, emulation could be used to automatically test protocol accuracy, to ensure the absence of deadlock, and to study the performance characteristics of the new protocol in the particular configuration. Finally, to facilitate selection of the next generation of communications gear, emulation can be used for hardware evaluation and software evaluation to answer the typical class of questions: 'Does a proposed change in communications equipment or protocols produce a cost-effective improvement in system perfor- mance and user satisfaction?'

Obviously, communications support and emulation have a great deal in common. Whether or not one is involved in the support of an actual communications environment or emulation of a virtual environment, the levels of interaction - device control, link control, process control - and the necessary support and service routines (e.g. statistics gather- ing) are virtually identical.

The remaining sections of this paper will describe the design and implementation of a communications software package to achieve the characteristics described above. The communications package, Commpac, addresses the multiple responsibilities of device control, data link control, and process level protocol support within a point-to-point com- munications environment. It will also handle the emulation of communications for studying hardware and software failures, and for performance evaluation of proposed hard- ware and software designs. Finally, the package is modular in design so that it may adapt to changes in system con- figuration and communication technology with a minimum of effort.

In summary, then, the structure of the Commpac system, while somewhat similar to many other multilevel communications packages (e.g. DEC's Comtex), differs in two fundamental respects:

it addresses the problems of communications support and communications emulation, it is highly modular in design, with standardized inter- module interfaces. This facilitates the conversion of individual components without affecting the remainder of the package.

The package discussed was originally implemented in the MACRO-11 assembly language and tested on a Digital Equip- ment Corp. PDP-11/45 computer at the University of Minnesota Computer Center. It is currently running and supporting communications at five other installations on a number of different DEC computers ranging in size from a PDP-11/03 to an 11/60. Copies of the complete package are available on request from the authors.

O V E R A L L SYSTEMS O R G A N I Z A T I O N

The communications package, Commpac, is constructed from independent subsystems which are ultimately linked together to form the executable program. The overall package currently contains five distinct subsystems:

• the executive, • the library,

vol 2 no 3 june 1979 99

Real or virtual -41------,D. Infor device

Real or Information ~ . virtual

process

Standardized d" ,~ slared

~ buffer

Commpac ~.~-~

F/gum 2. Overall system organization of Commpac

• device support subsystem, • information processor subsystem • operating system interface.

The executive provides the necessary scheduling, activation, synchronization, and control over all other subsystems, as well as performing initialization and post-mortem services. The library provides a set of routines needed by all other subsystems - for example, list handling, memory manage- ment, and data analysis procedures.

The last three subsystems listed above model the three fundamental levels of communication shown in Figure 1. The device support subsystem handles the control of a (real or virtual) communications line controller with any of the properties summarized in Part A of Table 1. This sub- system acts as a message producer, receiving (or creating) information from a (real or virtual) data path and passing it on to the next subsystem for processing. Conversely, when a response is ready for transmission this subsystem acts as a consumer of information, passing it on to the (real or virtual) data path. In support mode, the source/sink of a message to/from the device support module will be a hardware device connect to the line controller. In emulate mode the device support module can produce/consume the information stream directly, as long as it conforms to the message format con- ventions of the specific line controller being emulated.

The information processor subsystem handles the res- ponsibilities of the data link control protocol. This sub- system accepts messages arriving from the device support subsystem, interprets them according to the sematics of a particular protocol, encodes a response, and either transmits that response back to the device support level or passes the message on to yet another subsystem - the operating system interface.

The operating system interface subsystem handles and executes all operating system requests made from the infor- mation processor and, after appropriate action is taken, returns the appropriate response to the information pro- cessor. In support mode, the source or sink of the messages passed to or from the operating systems interface level will be actual user or system processes - e.g., application pro- grams, input/output support processes, or file handling routines. In emulate mode, this level can itself consume or produce messages to model the operational behaviour and performance characteristics of virtual processes or system devices.

All Commpac subsystems that need to exchange infor- mation communicate through standardized software inter- faces called shared tables and shared circular buffers. One subsystem asynchronously inserts information into the buffer by updating a buffer pointer and checking for over- flow. A second task removes information as it appears and updates the pointers accordingly. The format for in- formation is standardized, and so each subsystem is totally

transparent to the actual subsystem with which it is commun. cating. Those subsystems that activate one another but do not exchange messages (cf. Figure 2) do so via standardized calling sequences that are specified by Commpac and do not change. The standardization of information transfer provid- ed by these type of interfaces allows subsystems of the same class to be interchangeable. This interchangability is essen- tial to support or emulate a wide range of terminals, con- trollers, protocols, and operating systems without affecting the remainder of the package. The fundamental design goal of Commpac is to be able to replace an individual subsystem without unnecessary or unexpected changes to any other subsystem. This would allow us to achieve the flexibil i ty and multifunctionality which was described in the first sec- tion.

Figure 2 shows the overall system organization of the communication package, Commpac.

The intermodule links between the device support, in- formation processor and operating-system interface sub- systems are, as shown above, handled by shared buffers, pairs of buffer pointers and standardized message formats - the classic consumer/producer relationship (see Figure 3).

Should a buffer overrun condition occur, it is the res- ponsibility of the datalink control protocol (i.e. the infor- mation processor) to slow down the production of mes- sages into the shared buffer pool via one of the seven task types described in the next section.

T A S K S T R U C T U R E

The basic operational method of Commpac is as a task- driven package. All communications responsibilities, real or emulated, are defined in terms of the execution of a finite sequence of primitive communication tasks. The fundamental data structure within the package is the tasker list. This multilinked list contains the ordered list of tasks which are currently being performed by the system. Figure 4 shows the typical configuration of the tasker list during execution of some communications responsibility.

The executive subsystem contains the tasker control loop which continuously selects the next task on the tasker list

In .~..sb mi+3

rni÷2

m i . I

m/

Buf fe r pool

Figure 3. Intermodule Iinhs

, 4 - ~ Out

100 computer communications

Q o I-I---H:::,IT 1,14--I, ::IL I

Figure 4. Structure of the tasher list (A: null unused field)

and activates a service routine, within the appropriate sub- system, to carry out that task. With a sequential task (e.g. tasks A or D in Figure 4) the executive will wait until com- pletion of that task before going on to the next. With a concurrent task, (e.g. tasks B and C in Figure 4) both service routines will be initiated simultaneously.

There are seven task types which are recognized and handled by the executive. Their names, actions, and associated subsystems are summarized in Table 2. New tasks can be added to the tasker list or existing tasks removed by any of the Commpac subsystems during task execution. Tasks that initiate other tasks and then are deleted upon completion are called chained tasks. Modifications to the tasker list are handled by the list management routines, Insert, Delete, Replace, which are part of the library sub- system. So, for example, during execution of the ENCODE task service routine (cf. Table 2) in the information pro- cessor, we might place the newly completed message in the shared buffer between the device support and information processor subsystems, and then add a TRANSMIT task to the tasker list (this is called task chaining). When that task is activated the device support subsystem will extract that message from the buffer and transmit it to the (real/virtual) device connected to that controller. The source data for ENCODE is supplied by a concurrent READ task, again through a shared buffer.

Finally, tasks can be added to the tasker list after a speci- fied time delay instead of immediately. A set of scheduler macros will maintain a linked list of tasks and activation times. When the activation time is reached the task request will be moved to the tasker list and activated in turn. This will allow the modelling of time-dependent con- ditions, such as positive acknowledgement timeouts.

SUI~SYSTEMS

As mentioned earlier, Commpac is composed of five dis- tinct and independent sybsystems. This section will describe the responsibilities of these subsystems in more detail. The functions of the executive subsystem are:

• system initialization, • task initiation, • task scheduling, • console device support and command language inter-

pretation, • postmortem analysis, • system shutdown.

Upon entry to the system the executive subsystem activates a set of system initialization subroutines which initialize memory, the console device processor, and system vectors. User processes may be initialized by additional calls from the executive or from the console device processor. The tasker and scheduler lists are cleared and control is trans- ferred to the tasker control loop.

As described earlier, the primary function of the tasker and scheduler is to initiate tasks associated with an active process. A process may be defined as the set of tasks re- quired to support the activity of a device. As the tasker cycles through its list it initiates the service routine associat- ed with each task. The scheduler cycles through its list and moves any task whose activation time is reached to the tasker list.

The console device support function of the executive subsystem accepts Commpac input commands from a special, 'master user', input device called the console device. Commpac parses the command, performs the requested actions, and sends any output back to the console device. The command language of Commpac supports the dumping of memory and registers in numerous formats as well as the printing of a wide range of statistical summaries. Examples of Commpac command statements are shown in Table 3.

The postmortem facility prints out, on the console device, critical information pertinent to the conditions that caused the failure and which is necessary for its isolation and diagnosis.

The final service of the executive subsystem is the shut- down process. Devices activated by Commpac are de- activated and control is returned to the operating system.

The library subsystem contains conversion routines , string processing routines, list processing routines, and a memory pool manager. The subroutines of the library are available to all subsystems. The conversion routines are typically called by the console device to dump memory in one of a number of formats. The list processing routines facilitate the use of the standardized list structure of the entire system. String processing routines are used by the

Table 2. Seven basic task types in Commpac

Task Purpose Subsystem handling this task

READ

WRITE

TRANSMIT

RECEIVE

DECODE

ENCODE

WAIT

Input from a (real/virtual) operating system supported device Output to a (real/virtual) operating system supported device Transfer a message to the device support subsystem Enable input from the device support subsystem Analyse message received from device support sub- system Build a response message for the device support subsystem Monitor an interrupt driven task

Operating system interface

Operating system interface

Device support

Device support

Information processor

I nformation processor

Device support or information Processor

vol 2 no 3 june 1979 101

Table 3. Sample Commpac - command language statements

Command Meaning

RUN, COMM SEND, filename, host

RECEIVE, filename, host

CONNECT, host

DISCONNECT, host

DUMP, location, format

Activate the Commpac package Transmit the file called 'filename' Lo the indicated 'host' Direct input from the indicated 'host' to the file called 'filename' Perform all handshaking necessary to establish connection to the 'host' Terminate communication with 'host' Dump the contents of memory 'location' in the specified format

system to compare strings, move strings, and parse strings. The memory pool manager allocates memory to requesting routines from the memory pool and deallocates the memory upon release from the calling process.

The operating system interface subsystem is called from other subsystems to handle operating system requests. Centralizing all operating system requests into one sub- system makes the rest of Commpac independent of the particular operating system on which the communications package is running. The functions performed by the oper- ating system interface subsystem are:

• memory initialization, • high priority print to console device, • buffered console terminal input/output, • file processing, • transfer of current date, • transfer of current time, • transfer of control to the operating system.

The most important function of the operating system inter- face subsystem is file processing. This is handled by the READ/WRITE tasks within the subsystem, which operate concurrently with other chained tasks. READ and WRITE Interface with a particular operating system via the specific set of executive requests provided by a particular OS. The major jobs of the READ/WRITE tasks are:

• checking file specification accuracy (e.g. name syntax), • allocating sufficient buffer space, • activating an input/output request from a (real or

virtual) device via executive requests, • waiting for the completion signal.

The types of files supported will depend entirely on the service requests provided by the particular operating system.

Currently, the operating system interface subsystem has been designed to interface with the RT-| I operating system for the PDP-11 family of computers. Future plans call for the development of additional versions of this subsystem to interface with other operating systems. (e.g. Unix*) The modularity of Commpac will greatly facilitate this task.

The functions of the information processing subsystem are:

• setting up communications parameters, • requesting device initialization,

* Unix is a registered trademark of Bell Laboratoris

• decoding information from the device support subsystem, • routing information to the operating system interface

subsystem, • processing information received from the operating

system, • encoding messages for transmission to the device support

subsystem, • error processing and recovery, • handshaking, • message timing and size computation, • gathering statistics, • terminating the communication process, • sending console command messages, • routing console messages to the console device.

As mentioned before, the information processor interacts with the device support subsystem via a shared buffer and task chaining. Referring to Figure 5 - the RECEIVE task in the device support will typically chain to a DECODE task handled by the information processor. The information processor decodes the message and encodes a response according to the rules of the protocol being implemented. The information processor also interacts with the operating system interface subsystem. During the chained TRANSM IT/RECEIVE/DECODE/ENCODE sequence, two concurrent tasks are in effect - READ and WRITE. These tasks provide message to or from operating system support- ed processes and/or devices. The information processor and the operating system interface communicate via a second shared buffer.

In the communications system, the information processor may be thought of as the implementation of a datalink protocol. Currently, our information processing subsystem implements the protocol termed DDCMP 3. However, in a larger context, the information processor could be oriented toward noncommunications objectives. In a laboratory con- trol system an analogue device may be used in an experiment. The information processor would analyse the incoming data, decode it, issue a control function back to the device sup- port subsystem, and store the decoded information on a mass storage device via the operating system interface.

The function of the device support subsystem in the com- munications application is to support the hardware com- munications controller. This involves:

• setting up the controller, • transmission of data (in both half- and-full-duplex modes), • reception of messages, • error detection and recovery, • passing of messages to and from appropriate information

processing tasks, • gathering of statistics, • trapping communications data for debugging.

The interval timer within the Commpac system can discrim- inate between intervals to within 10/us. This would allow

Figure 5. Typical tasker l ist chain wi th concurrent tasks

102 computer communications

. . . . I TRANSMIT ] ~ - ~ ~ ] RECEIVE J

Figure 6. Modified tasher list chain

(theoretically, at least) the support of data transmission rates of up to 100 000 bit/s.

The actual rate of message flow between the device sup- port and information processor subsystems in a support en- vironment is, of course, determined by the actual devices and their users. However, in the emulate mode the rate of mes- sage flow between these two subsystems can be controlled via the WAIT task, which forces a timed delay in task pro- cessing. Thus, for example, to emulate a slowdown in the message input rate, we would simply have the TRANSMIT task chained to a WAIT task, thus modifying Figure 5 to look like Figure 6.

Two functionally equivalent device support subsystems have been written for two types of synchronous controllers -Digi tal Equipment Corp. DP-11 and DU-114. Since the interfaces with the rest of the system are standardized, either one of the versions can be used without affecting changes to the remainder of the package. Future versions of Commpac may include support of differing controllers and protocols as the services and needs of the marketplace change.

EXAMPLES

The Commpac system is currently being used for support, emulation, and diagnosis of a wide range of communications functions. At one site the system supports a remote batch terminal called a Univac 1004. The terminal can dial into a host computer system with autoanswer capabilities and communicate via the Commpac system. Listings can be printed on the terminal and data cards can be transmitted back to the host.

The same system is also being used to emulate a remote batch terminal in order to quickly allow a minicomputer to communicate with a host using the DDCMP protocol. The development time for the new DDCMP information-processing module was about 6 man weeks. The configuration for the emulation is being distributed to PDP-11 computer systems in the University of Minnesota community. This version of the system is operational on five computer systems rang- ing from the PDP 11/03 microcomputer to the PDP 11/60.

The system is also being used by the engineering staff to diagnose failing Univac 1004 terminals. The system allows the engineer to control the transmission of messages to the terminal. The engineer can follow the messages through the hardware until the failure is found and corrected.

The Univac 1004 protocol module and the DDCMP emulation module are both information processing sub- systems. The remaining four subsystems within Commpac require no changes to accommodate the specific information processing subsystem being used. In both cases the Commpac user enters a command which prepares the system for communication and initiates a connection from the terminal end of the data link. The system will exchange the handshaking messages and be ready for data transfer. To send or receive files, the user enters a SEND and/or RECEIVE command (cf. Table 3) specifying the appropriate device and filenames. The operating system interface sub- system queues the file names until it is time to service them. After the handshaking sequence is completed the concur- rent READ and WRITE tasks are initiated. Files are approp-

riately opened or initialized. The READ task tried to fill the shared buffer and the WRITE task writes out each full block until the transfer is complete and the files are closed. If more files remain to be processed the above procedure is repeated. While the READ task is providing data, the infor- mation processing subsystem is removing data and encoding it into the appropriate protocol specification. The message is then placed into another shared buffer for transmission. The ENCODE task is deleted and a TRANSMIT task is entered into the tasker list. This task transmits the message and then chains to the RECEIVE task. A timer is activated, and if a response is not received before the timer expires, error recovery processing is initiated according to the semantics of the information-processing subsystem currently being supported. In the case of the DDCMP module, this error detection/correction mechanism involves the use of a CRC-16 check character, an ARQ acknowledge/not- acknowledge scheme, and timeouts, as described in Refer- ence 2. All message-level error detection and error correc- tion will be contained within the information processor as part of the DLC protocol specification. If the response message is received the WAIT task chains to the DECODE tasi The transmission and reception until this point have been performed by the device support subsystem. The DECODE task service routine of the information processing subsys- tem verifies that the incoming message in the shared buffer has been received without error, and if so, translates the message and places the data in the shared buffer for the WRITE task to process. If errors are detected the appropriate error recovery processing is initiated.

For a description of other current usages of Commpac as well as additional details on other experiments which we have conducted using this package, see Reference 5.

This technical report contains specific implementation details on each of the five subsystems, their interfaces, and the specific modules that have been implemented so far. There are currently plans to develop and expand the range of modules to include the

• device support level: DZ-11 multiplexer, • datalink level: HDLC/SDLC bit-oriented protocols, • operating-system level: Unix.

CONCLUSIONS

The communication system described in this paper was designed to handle a wide range of communications related tasks, including both support and emulation. In addition, it was designed to adapt, with miniminal effort, to changes in system configuration - either software or hardware. Since it was first available in February, 1978, the system has been brought up on a number of different computers in the PDP-11 family (e.g., PDP-11/03, 40, 45, 60) and has responded fairly favourably to a number of different responsibilities and environments. This has been due to the modular design of Commpac. The interchange- ability of subsystems has allowed Commpac to easily adapt to new operating systems, device controllers, and protocols.

ACK NOW LEDG EM ENTS

This work was partially supported by the University of Minnesota Computer Center and US National Science Foundation Grant MCS75-09789.

vol 2 no'3 june 1979 103

REFERENCES

Pouzin, L and Zimmerman, H 'A tutorial on communi- cations network protocols' Proc. IEEE Vol 66 No 11 (November 1978)

Pouzin, L 'Basic elements of a network data link control' Comput. Commun. Rev. Vol 5 No 1 (January 1976)

Wecker, S 'The design of DECNET - a general purpose network base' Electro 76 Conf. Boston, USA (1976)

Peripherals handbook Digital Equipment Corporation, USA (I 978)

May, L E 'A multilevel modular structure for support and emulation in communications' University Computer Center Technical Report University of Minnesota, USA (January 1979)

and inicrosystems the authoritative international journal on microcomputer technology and applications for designers

Special individual subscription rate only £15 a year

Further details and subscription rates from:

David Burr, IPC Science and Technology Press Ltd, Westbury House, Bury Street, Guildford, Surrey GU2 SAW England Telephone: 0483 31261 Telex: 859556 Scitec G

~ T

4

104 computer communications