Incremental Design Migration Support in Industrial Control ......Acknowledgements I would like to...

Incremental Design Migration Support in Industrial

Control Systems Development

Harish Balasubramanian

Thesis submitted to the Faculty of the

Virginia Polytechnic Institute and State University

in partial fulfillment of the requirements for the degree of

Master of Science

in

Computer Engineering

Cameron D. Patterson, Chair

William T. Baumann

Michael S. Hsiao

December 02, 2014

Blacksburg, Virginia

Keywords: Industrial control systems, incremental migration, model-based design, interface

abstraction

Copyright 2014, Harish Balasubramanian

Incremental Design Migration Support in Industrial Control Systems

Development

Harish Balasubramanian

(ABSTRACT)

Industrial control systems (ICS) play an extremely important role in the world around us.

They have helped in reducing human effort and contributed to automation of processes in

oil refining, power generation, food and beverage and production lines. With advancement

in technology, embedded platforms have emerged as ideal platforms for implementation of

such ICSes. Traditional approaches in ICS design involve switching from a model or mod-

eling environment directly to a real-world implementation. Errors have the potential to go

unnoticed in the modeling environment and have a tendency to affect real control systems.

Current models for error identification are complex and affect the design process of ICS

appreciably. This thesis adds an additional layer to ICS design: an Interface Abstraction

Process (IAP). IAP helps in incremental migration from a modeling environment to a real

physical environment by supporting intermediate design versions. Implementation of the

IAP is simple and independent of control system complexity. Early error identification is

possible since intermediate versions are supported. Existing control system designs can be

modified minimally to facilitate the addition of an extra layer. The overhead of adding the

IAP is measured and analysed.

With early validation, actual behavior of the ICS in the real physical setting matches the

expected behavior in the modeling environment. This approach to ICS design adds a signif-

icant amount of latency to existing ICSes without affecting the design process significantly.

Since the IAP helps in early design validation, it can be removed before deployment in the

real-world.

Dedication

To my parents and teachers who made me what I am today

iii

Acknowledgements

I would like to express my gratitude to my academic advisor Dr. Cameron Patterson for

providing me the opportunity to work under his guidance. His advice and constant support

has been valuable in completing my thesis. Dr. Patterson, it has been an honor and great

pleasure to work with you and I am grateful for your patience. I would like to thank Dr.

William Baumann and Dr. Michael Hsiao for their participation in my thesis committee.

Thanks to Zane R. Franklin for his guidance and support in my research and writing my

thesis.

Thanks to my father, P.M. Balasubramanian and my mother, Padma Balasubramanian, for

their support and guidance. A special thanks to all my roommates and friends who have

been an incredible support during my research and writing.

This material is based upon work supported by the National Science Foundation under

Grant Number CNS-1222656. Any opinions, findings, and conclusions or recommendations

expressed in this material are those of the author and do not necessarily reflect the views of

the National Science Foundation.

The ZedBoard and design tools used were donated by Xilinx, Inc.

iv

Contents

List of Figures viii

List of Tables x

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 IAP Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2 IAP Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Industrial Control Systems 6

2.1 Process Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1.1 Basic Components of a Control System . . . . . . . . . . . . . . . . . 7

2.1.2 Open and Closed Loop Control Systems . . . . . . . . . . . . . . . . 8

2.1.3 Steps in Control System Design . . . . . . . . . . . . . . . . . . . . . 9

2.1.4 Platforms used in Control System Implementation . . . . . . . . . . . 10

2.1.5 Embedded Platforms for Control Systems . . . . . . . . . . . . . . . 12

2.2 Model-Based Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Embedded Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4 Simulation Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

v

3 Interprocess Communication 17

3.1 Need for IPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.2 IPC Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2.1 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2.2 Message Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.2.3 Shared Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.4 Mapped Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.5 Pipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2.6 Named Pipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.7 Summary of IPC methods . . . . . . . . . . . . . . . . . . . . . . . . 24

3.3 Sockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3.1 Socket Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.3.2 Creating and Destroying Sockets . . . . . . . . . . . . . . . . . . . . 27

3.3.3 Bind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.4 Sending Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.3.5 UDP Sockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.4 Design Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4.1 Named Pipes for IPC . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4.2 UDP over TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4.3 Data and Memory Handling using Structures . . . . . . . . . . . . . . 33

4 Design Implementation 35

4.1 Incremental Design Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.2 RCMP Design Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2.1 Two C Processes Communicating with Named Pipes . . . . . . . . . 38

4.2.2 One C Process and One MATLAB Process Communicating with Named

Pipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.2.3 UDP Sockets - Server and Client . . . . . . . . . . . . . . . . . . . . 40

4.2.4 One C Process and IAP . . . . . . . . . . . . . . . . . . . . . . . . . 40

vi

4.2.5 RCMP Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5 Results 45

5.1 Parameters used to Measure System Performance . . . . . . . . . . . . . . . 45

5.2 Obtaining Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2.1 Function to Obtain Timestamp and Unix time . . . . . . . . . . . . . 46

5.2.2 Timestamp of the Processes without Synchronisation . . . . . . . . . 47

5.3 Network Time Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.3.1 Need for NTP and Drift . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.3.2 Clock Synchronization using NTP . . . . . . . . . . . . . . . . . . . . 50

5.3.3 Timestamp of the Processes with Synchronisation . . . . . . . . . . . 53

5.4 Delay Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.4.1 Experimental Assumptions when Measuring Delay . . . . . . . . . . . 55

5.4.2 Measuring Delay in the Loop without IAP . . . . . . . . . . . . . . . 56

5.4.3 Measuring Delay in the Loop with IAP . . . . . . . . . . . . . . . . . 57

5.5 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.5.1 Computation Delay with and without IAP . . . . . . . . . . . . . . . 59

5.5.2 Communication Delay with and without IAP . . . . . . . . . . . . . . 60

5.5.3 Control Cycle Delay with and without the IAP . . . . . . . . . . . . 61

6 Conclusions 65

6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

6.3 Future Work and Recommendations . . . . . . . . . . . . . . . . . . . . . . . 66

Bibliography 68

A Code used in the Design - without IAP 71

B Code used in the Design - with IAP 76

vii

List of Figures

1.1 Interaction between plant and controller without IAP . . . . . . . . . . . . . 4

1.2 Interaction between plant and controller with IAP . . . . . . . . . . . . . . . 5

2.1 Basic components of a control system . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Steps in design of a control system . . . . . . . . . . . . . . . . . . . . . . . 11

3.1 IPC in UNIX systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Pipe in a single process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3 Pipe in a single process after forking . . . . . . . . . . . . . . . . . . . . . . 22

3.4 Two processes communicating with two FIFOs . . . . . . . . . . . . . . . . . 24

3.5 Deadlock in named pipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.6 UDP socket connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.7 Frame used in the design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1 Incremental migration from fully-modeled to fully-implemented control system 37

4.2 Two C processes communicating with named pipes . . . . . . . . . . . . . . 38

4.3 One C process and one MATLAB process communicating with named pipes 39

4.4 Server and client communicating over UDP sockets . . . . . . . . . . . . . . 41

4.5 One C process and IAP communicating with named pipes and UDP sockets 41

4.6 RCMP implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1 Processes where timestamp is recorded . . . . . . . . . . . . . . . . . . . . . 49

5.2 Hierarchical levels of clocks . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

viii

5.3 Synchronization setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4 Control cycle loop with IAP . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5.5 Computation delay of controller with and without IAP . . . . . . . . . . . . 61

5.6 Computation delay of plant with and without IAP . . . . . . . . . . . . . . . 62

5.7 Communication delay with and without IAP . . . . . . . . . . . . . . . . . . 63

5.8 Total delay in one control cycle loop with and without IAP . . . . . . . . . . 63

5.9 Percent increase in total delay in one control cycle loop after addition of an

IAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

ix

List of Tables

3.1 Advantages and disadvantages of different IPC methods . . . . . . . . . . . . 26

3.2 Comparison of TCP and UDP sockets . . . . . . . . . . . . . . . . . . . . . . 31

4.1 RCMP data and memory handling . . . . . . . . . . . . . . . . . . . . . . . 44

5.1 Timestamp of the processes before synchronisation - iteration 1 . . . . . . . 48

5.2 Timestamp of the processes without synchronisation - iteration 2 . . . . . . 48

5.3 Timestamp of the processes after synchronisation - iteration 1 . . . . . . . . 53

5.4 Timestamp of the processes after synchronisation - iteration 2 . . . . . . . . 53

5.5 Mean and standard deviation of delay components without IAP over 50 iter-

ations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.6 Mean and standard deviation of delay components with IAP over 50 iterations 60

x

Chapter 1

Introduction

A control system is a mechanism by which an input fed into the system affects, regulates

or maintains the output of the system. The necessary control action can be achieved by a

single device or a set of devices acting together to create a desired output. There are many

examples of control systems in the world around us. A thermostat is an example of a control

system where the the desired temperature is achieved through a simple knob and changing

the position of the knob controls the temperature. The main aim of any control system is

to reduce human effort and eliminate human error. Human error can play a significant role

in aircraft control or in a nuclear reactor where small mistakes can lead to disaster.

As control systems become more complicated, the need for advanced computers becomes

imperative. The computers which are part of a control system need to be fast, accurate

and simple in their design. Embedded platforms provide an ideal environment to implement

control systems since they are designed to perform a specific task and can be customized

to suit a particular need. The design of processors, which drive any computer system, is

being improved to increase their speed and accuracy. The interaction between an embedded

platform and a control system is key to effective control action.

1

Harish Balasubramanian Chapter 1. Introduction 2

1.1 Motivation

The existing approach to design control system begins in a modelling environment before

moving to a real physical environment and is called model-based design. In the actual

design process, there are incremental steps in between the modelled versions to the real

physical version. Incrementally migrating towards the real physical system is helpful in early

validation of the overall design and identifying the incorrect components of the system in

case of deviation from expected behaviour of the system. Typically, the expected behaviour

of the various components of the control system are established at the end of the modelling

step and serves as the benchmark against which the behaviour of the real components are

compared. Communication between the different components is vital because the step-by-

step migration process requires different communication interfaces and protocols.

The previous section described the prevalence of control systems and the importance of

an embedded platform in control system design. Their interaction with one another plays a

significant role to get the best possible control action. Since both control systems and embed-

ded platforms need to be fast and reliable, the communication or interaction between them

needs to add minimal or no overhead to the overall system designs. Though fast interfaces

are available, communication can become a bottleneck and tends to occur when there is no

flow control or redundant communication occurs. Without flow control, one system becomes

overwhelmed by the other resulting in stalling and slow system performance. Hence, care

must be taken that the different systems coordinate with each other and wait for one another

to communicate. Redundant communication results in the payload being resent which can

block the communication channel and throttle system performance. Communication must

be controlled and accurate to prevent bottlenecks. Incorrect communication mechanisms

results in incorrect values used to control and maintain the system resulting in unstable

performance and failure to obtain the desired result. Thus, maintaining accuracy in control

system communication between the various components is critical. The thesis deals mainly

with improving and evaluating communication between two systems: an embedded platform


and a computer model replicating a control system.

1.2 Contributions

In order to better facilitate communication for the incremental design process of control

system, an additional process was added: an Interface Abstraction Process (IAP). The IAP

removes the complexity involved in the communication channel between the components and

provides seamless connection. The following sections explain the features of the IAP and its

implementation.

1.2.1 IAP Features

The features of the IAP are listed below:

1. The design of the IAP is straightforward and does not create additional complexity to

the overall control system design.

2. IAP supports the incremental modes in the design process and is helpful in faultless

migration to the real physical system from the modelling environment.

3. The IAP is autonomous and latency added is measured and analyzed. If the amount

of latency is significant then the IAP is removed before deployment. Since the IAP is

added as an additional layer on top of existing control systems it is easy to remove the

IAP from the design.

4. The flow control between the various components is maintained which guarantees that

none of the components are overwhelmed or throttled.

5. The IAP is portable across different platforms and there is no need for code redevel-

opment.


1.2.2 IAP Implementation

The two main components of any control system are a plant and a controller and there exists

a close interaction between the two. The controller generates control signals for the plant

which uses the control signals to control or stabilize a process parameter. The interaction

between a plant and controller is bi-directional in cases where the controller depends on the

previous output of the plant to generate the current control signals. Hence, such systems

involve feedback and Figure 1.1 shows the most basic form of interaction in a control system.

A detailed explanation of control system components is given in Section 2.1.

Controller Plant

Control SignalsSetpoint

Sensor Signals

Figure 1.1: Interaction between plant and controller without IAP

To enhance the interaction between a plant and controller, the IAP was added as an ad-

ditional layer or process. It is a simply a layer which abstracts the interfaces that are

communicating and acts as a bridge to help in easier communication and does not add any

complexity to existing design of a control systems. The IAP also helps by supporting the

intermediate design versions when migrating incrementally from a modelling to a real de-

sign environment. Incremental design versions help in easier verification and validation and

aids in faster error identification. The location of the IAP in a control system is shown in

Figure 1.2.


Controller Plant

Control Signals

Setpoint

Sensor Signals

IAP IAP

Figure 1.2: Interaction between plant and controller with IAP

1.3 Thesis Organization

This thesis is organized as follows: Chapter 1 discusses the motivation and contributions of

this work. Chapter 2 describes industrial control systems and existing design steps along

with target platform used in the implementation. Chapter 3 provides an overview on inter-

process communication (IPC), sockets and design decisions for the implementation. Chapter

4 explains the detailed steps involved in the implementation. Results and analysis are pre-

sented in Chapter 5. Chapter 6 summarizes the thesis and provides scope for future work

and improvement.

Chapter 2

Industrial Control Systems

This chapter provides a detailed overview of ICS, the steps in their design and commonly

used platforms in designing control systems in Section 2.1. Section 2.2 gives an overview

of model-based design and existing approach to control system design. Section 2.3 explains

the features of the embedded platform used in the study. Section 2.4 discusses modeling

environment used.

2.1 Process Control Systems

The fundamental element in an ICS is a process, which is a is a sequence of steps to obtain

end products using raw materials. Process control is controlling the parameters used in the

sequence of steps to get maximum yield with the available raw material. It is required in

oil refining, food and beverage industry, water treatment, production line or any process

where external factors such as flow, temperature, pressure and level can affect the yield.

For example, in an oil refinery, temperature and pressure need to be controlled to get the

maximum quantity of the required end product. Automation enables operating personnel to

operate a complex process remotely.

6

Harish Balasubramanian Chapter 2. ICS 7

Process control is done mainly for three reasons:

1. Quality control: Process control ensures product quality to the preset specification and

standard during the manufacturing process.

2. Optimizing productivity: Process control optimizes the relationship between input and

output by controlling process parameters. Optimization refers to getting maximum

output from minimum input.

3. Safety control: An unstable process can result in loss of property and life and can be

dangerous to the environment. To avoid such damage, manufacturers have to maintain

process parameters within safe limits.

A process variable is a condition of the process which can affect the process under con-

trol. Examples of process variables are pressure, temperature, density, flow, mass, etc. The

setpoint is the value for a process variable that is desired to be maintained. Error is the

difference between a process variable and the setpoint, and can be positive or negative. The

objective of any process control system is to minimize the error. The magnitude of error at

any time is measured and compared to the previous error value and provides the basis for

determining the change in error over time.

2.1.1 Basic Components of a Control System

The basic components of a control system are shown in Figure 2.1 [1]. The input to a

control system is a measured input signal which is an input from the plant or an instrument.

The setpoint is the desired input value. Reference signal, I(x), is the difference between

the measured input signal and the setpoint. Error detector is an element which computes

the difference between the feedback signal and reference signal to obtain the error signal,

E(x). The controller is an element which tries to minimize the error signal by generating

the appropriate control signal, C(x). The control signal works as an actuating signal for the

plant which produces desired control variable, Y (x). The plant is a device which operates


on the control signal which in turn controls the process variable or parameter. A feedback

element is a function of the controlled variable which is compared to the reference signal by

the error detector [1].

Setpoint

Measured

input

signal +

+/-

Reference

signal

Error

detector

Error

signal

Controller Plant

Control

signal

Controlled

variable

Output

element

Feedback

loop

I(x) E(x) C(x) Y(x)

Figure 2.1: Basic components of a control system

2.1.2 Open and Closed Loop Control Systems

An open-loop control system is one where the output is directly related to the input and

does not have a feedback mechanism to regulate the input via an output. The output

is not compared with the setpoint, and is used in simple processes where a mathematical

relationship exists between the input and output. An example is a conventional oven where

inputs are controlled by a human operator.

A closed-loop control system is one where the output is related to the input and the previous

output via a feedback controller. The current output signal is the difference between the

input and previous output until it matches the setpoint. The feedback controller tries to

minimize the error signal which is the difference between the measured variable and the


setpoint. An example of a closed-loop control system is the cruise control mechanism of a

car. The speed of the car is set and depending on the current road conditions, the speed is

increased or decreased to match the set speed [1].

2.1.3 Steps in Control System Design

The traditional steps in designing a control system are shown in Figure 2.2. The sequence

of steps are described below [2].

1. System specification is the minimum system requirement to configure a plant with

control software. It is required for compatibility of the hardware with the software.

The system specification for a logic-based controller would include the number of I/O

cards and communication interface method.

2. Plant modeling is an accurate software prototype for the simulation model of the

physical plant to be controlled. It is required for the closed-loop simulation of a plant

in accordance with control logic. A plant model is validated using experimental data

obtained with the plant in open-loop which is essential as the controller design is based

on the response of the plant model.

3. Controller design is a process of designing a tuning method for control variables. Con-

trolled variables can be tuned by proportional for present error, integral as accumu-

lation of past errors, and derivative as prediction of future errors or a combination of

these methods using the error between actual and desired parameter of the plant as

feedback.

4. Following controller design, verification is performed in two steps. Hardware-in-the-

loop (HIL) simulation is where the controller is loaded on prototype hardware and

tested with the simulation model of the plant in closed-loop. This is followed by

verification of the controller with the actual plant in closed-loop and measuring the

actual performance of the plant. Any difference between the actual performance of the


plant and the required performance is compensated by tuning the controller. The final

controller performance is limited by hardware capabilities, available control effort, and

the cost of implementation.

5. After the results are obtained from control verification, they are validated using simu-

lation and any discrepancy leads to revision of the plant and controller model. Addi-

tionally, this step also performs cost analysis.

6. Deployment is the final stage in the design process where the prototype system is

converted into an automated plant system by configuring the software controller for

the hardware plant.

The next section describes commonly used platforms to implement control systems and

justifies the emergence of embedded platforms for ICS design.

2.1.4 Platforms used in Control System Implementation

Programmable Logic Controllers (PLC), Supervisory Control and Data Acquisition (SCADA)

and Distributed Control Systems (DCS) are commonly used organizations for control sys-

tems [3]. A PLC is an automated system designed to control the process variables through

sequential logic and is a digital technology for process control. All PLC’s consist of a power

supply, central processing unit, memory, software and communication interfaces. The ad-

vantage of using a PLC is that they can withstand harsh environments and are capable of

communicating efficiently. The central processing unit in a PLC consists of a microprocessor,

memory and other integrated circuits to monitor logic and maintain communication. There

are two modes of operation in a PLC: the programming mode in which a logic sequence is

downloaded from a computer, and the running mode in which the PLC executes the logic

sequence which it downloaded. PLCs can interact with multiple inputs and outputs at the

same time and they can be reconfigured. PLCs communicate with the real world with the

help of input and output cards. Input cards accept analog or digital inputs from the real


System

specification

Plant

modeling

Controller

design

Controller

verification

Design

revision

Deployment

Figure 2.2: Steps in design of a control system

world and converts it into binary values. Output card accepts binary values which can be

used to operate an analog or digital device. A digital output to a device turns the device

such as a motor or valve, ON or OFF. A digital signal to an analog device is converted into

a corresponding voltage or current such as a flow meter or temperature regulator. PLCs are

also used for batch control systems. For example, in a bottling plant, the cap is not fitted

in the bottle until liquid is filled to a certain level in the bottle. PLCs are used as safety

interlock systems as they can implement decisions in a short time period [4].


A SCADA system is a system of nodes used in ICS commonly used for supervisory functions.

The key function of any SCADA system is data acquisition which is used to store data

from the plant. Data is presented to the outside world using a Human Machine Interface

(HMI). The HMI unit facilitates the display of control variables and allows operators to make

changes to control system behaviour. An important advantage of using SCADA systems is

that they can be geographically distributed across a wide area. The hierarchial organization

of a SCADA system consist of Remote Terminal Units (RTUs) and Master Terminal Units

(MTUs). An RTU is typically a specialized PLC which communicates to an MTU, which

is an interface to the outside world. Communication between RTUs and MTUs must be

efficient and reliable since RTUs are geographically isolated over long distances. SCADA

systems are used for event-driven control rather than continuous monitoring of a control

system. An example of an event-driven control would be when the speed of a motor drops

below a particular threshold value compared to continuous monitoring of motor speed. This

significantly reduces the amount of communication to RTUs [5].

DCSes are control systems with control distributed throughout the system, and are com-

posed of elements which are linked via network protocol to many controllers. In terms of

architecture, a DCS resembles a SCADA system but they differ in functionality. A DCS

is typically used in monitoring control variables in a plant whereas a SCADA system is

used in event-driven actions. This makes DCS more suitable for interconnected local plants

rather than geographically distributed systems. DCSes have more reliable communication

mechanisms compared to SCADA since there is continuous stream of data sent from a plant.

DCSes are also used for maintenance of control systems within a local plant [5].

2.1.5 Embedded Platforms for Control Systems

The design of process control systems on embedded platforms is found in a wide range of

applications such as consumer electronics, medical equipment, navigation systems and indus-

trial control systems. Embedded platforms are typically microcontrollers, microprocessors,


Digital Signal Processors (DSPs) and Field Programmable Gate Arrays (FPGAs). The ad-

vantage of using embedded platforms for control system design is that they can be optimized

to reduce the size and cost of the system and they can be mass-produced easily. FPGAs have

configurable hardware logic blocks which can implement multiple control loops independent

of each other resulting in concurrent processing. Also, FPGAs have the added advantage of

being able to be rewired or reconfigured at any stage in the design process. Embedded plat-

forms support a wide range of peripherals such as I/O, memory and analog/digital converters

which are used in control system design. Recent developments in industry have led to a hy-

brid design which combine both configurable logic blocks and interconnects together with

an embedded processor such as ARM. The software part of a design is implemented on the

embedded processor and the hardware part is implemented on the configurable logic blocks.

Thus, overall system design becomes simpler as both hardware and software components can

be designed on the same chip.

The design flow of embedded platforms is traditionally done in software or hardware de-

scriptive languages which deal with logic gates, registers or bits [6]. But for traditional

control system design, modeling and simulation environments such as MATLAB/Simulink

[7] and LabVIEW [8] are used. Recently, additional libraries or tools are provided by such

simulation environments to aid in embedded system design. Examples of such tools are DSP

Builder [9] or MATLAB Embedded Coder [10]. The Embedded Coder, for example, gen-

erates C/C++ code from Simulink models or MATLAB functions that can be implemented

on embedded platforms or any supported targets directly. Embedded Coder also supports

a wide range of embedded platforms and processors. With these tools, the control systems

which are described in a modeling environment are custom-made and converted to suit the

needs of the embedded platform.

The next section describes how control systems are designed in a modeling environment.


2.2 Model-Based Design

Model-Based Design (MBD) is a mathematical and visual method of addressing problems

associated with designing embedded software for complex control systems [11]. MBD pro-

vides a platform for controller development based on a mathematical model of a plant which

has advanced features and aids in rapid design and verification. Plant behaviour is anal-

ysed and a set of equations are generated. A controller is designed with the help of these

equations [12]. Following the design step, both the controller and plant are simulated and

modified until they match expected behaviour. The advantage of MBD is that design re-use

is possible and it makes the development process easier.

The plant in a control system refers to the system, device or process which one intends

to control. Before its deployment any plant in a control system environment undergoes a

thorough design verification as described in the previous section. The control systems being

designed today suit cross-functional domains which increases their complexity, thus making

them harder to verify and test. Moreover, the traditional approach to plant verification

requires actual hardware and begins late in the design cycle. To overcome these challenges,

MBD consists of a system simulation step, in which engineers create an accurate model of the

plant so that extensive testing can be done on it. The model can be used to test the actual

control algorithm which ensures design errors are observed earlier and necessary corrective

action can be taken. The actual plant hardware and control software is done only after

system simulation verifies the plant model completely. In addition, system simulation also

enable real-time testing of the control algorithm. The plant model is used to generate code

which is run on a computer and real-time plant behaviour can be observed. The controller in

a control system is the element which can measure and modify the behavior of the device or

process under control. The controller can be modeled in software or hardware. The controller

provides a set of instructions which when altered modifies the way in which control system

interacts and performs.

The existing approach to designing control systems consists of switching from a Modeled


Controller and Modeled Plant (MCMP) to a Real Controller and Real Plant (RCRP). The

controller can either be a controller model or an embedded controller. The plant can either

be a plant model or a real physical plant. The proposed design consists of a controller and

plant that are connected by an Interface Abstraction Process (IAP) which acts as a bridge

to connect both ends. The IAP also supports incremental migration such as Real Controller

and Modeled Plant (RCMP) or a Modeled Controller and Real Plant (MCRP). An example

of the RCMP implemetation will be detailed in Chapter 4.

2.3 Embedded Platform

The embedded platform used in the design is the ZedBoard which uses the Xilinx Zynq-7000

All Programmable SoC [13]. The Zynq-7000 consists of a dual-core ARM Cortex-A9 whose

operating frequency is 667 MHz together with a Xilinx Artix-7 FPGA. The Zynq supports

Linux, Android and any RTOS environment. It also contains on-board USB-OTG and USB-

UART support. The Gigabit Ethernet port has a static IP address of 192.168.1.10. The

ZedBoard memory consists of 512 MB DDR3 memory and a 4 GB SD card. The OS chosen

was Linux since it is well supported by Xilinx and serves the purpose of IPC. The version of

Linux used is Zynq Release 14.4 and it is booted from the SD card. This version of Linux

contains a ramdisk image which supports SSH by default [14].

2.4 Simulation Environment

The simulation environment is needed in the design to create a plant and/or controller model.

The simulation platform chosen is MATLAB due to its popularity and widespread adoption.

In addition, Simulink has a block diagram-based approach which is easier to understand and

interpret. The version of MATLAB used is 64-bit 8.0.0.783 R2012b [15]. MATLAB is run

on a laptop with Intel processor.


This concludes the ICS overview. ICS design steps and platforms were discussed along with

the embedded platform and simulation environment used in our work. The next chapter

deals with IPC mechanisms for IAP interaction.

Chapter 3

Interprocess Communication

Section 3.1 explains the need for IPC and Section 3.2 lists the various options of IPC avail-

able and their features. Section 3.3 gives an overview of sockets. The design decisions for

communication are justified in Section 3.4. Section 3.4.3 explains how data is handled and

memory is allocated in the system.

3.1 Need for IPC

The need for IPC in our experiment arises from the fact that the design adds an extra process

or layer to an existing control system design. In order for these processes to communicate

with each other and share data, a study was done to choose the best method of IPC for

our experiment. IPC refers to a set of mechanisms or methods supported by an operating

system allowing one process to communicate with another process. The two processes can

be running on the same system or on different systems that are connected in a network [16].

IPC provides a medium where data exchange can occur between two processes.

IPC mechanisms differ in the following criteria:

• Whether communication is supported on the same computer or with a networked

17

Harish Balasubramanian Chapter 3. Interprocess Communication 18

computer.

• The number of processes that can communicate.

• The level of synchronization provided by the processes and the IPC mechanism.

• Where the IPC mechanism resides in the system, outside or inside the kernel or within

the file system.

In traditional UNIX systems, the various ways for processes to communicate with each other

are summarized in Figure 3.1 [17].

Kernel

Process 1 Process 2

Shared

information

Filesystem

Process 2Process 1 Process 1 Process 2Shared

memory

Figure 3.1: IPC in UNIX systems

1. The first type consists of processes sharing data with each other through a file in the

file system. The processes must go through the kernel to access the data in the file

system. Files are an example of this type of IPC.

2. The second type consists of processes sharing data that resides within the kernel.

System calls are used to access the shared information in the kernel. Pipes, named

pipes and message queues are an example of this type.

3. The third type consists of processes which setup a region of memory for sharing data.

After the kernel helps in setting up the region of memory, data is shared without

involving the kernel. Shared memory is an example of this type.


3.2 IPC Types

The following section enumerates the different IPC mechanisms explored along with their

advantages and disadvantages.

3.2.1 Files

Two processes can communicate via a file that exists in the filesystem. One process writes

data into the file and a second process reads this data. The read and write operations cannot

occur at the same time, hence one process always writes data first and then another process

reads it. This is the simplest method of IPC but it tends to be slower than other IPC

mechanisms and uses up space on the file system. When a file write is done, it is written

to memory and then moved to disk. Whenever a file access is done, it is fetched from the

disk and this adds overhead to the communication. To access a file in the file system, two

operations have to be done.

1. Invoke system calls such as open, read, write and close.

2. I/O operations for accessing the disk.

3.2.2 Message Queues

A message queue is a service provided by the operating system and is a unidirectional stream

of messages. They can be used as a method of IPC for transferring data from one process

to another. In a message queue, every message has a type field that indicates priority and

messages can be read from the queue depending on the priority. Generally, one process

creates the queue and other processes which wish to communicate with the process connect

to this queue. In a message queue, writing and reading are independent events. Message

queues are removed when processes are terminated [18].


3.2.3 Shared Memory

Shared memory is the fastest form of IPC available and is a mechanism by which two

processes share a block of memory. The shared memory block is allocated by only one

process. Any other processes which wish to communicate can access the memory block that

was allocated. The shared memory block does not reside in the kernel thus eliminating the

need for system calls to access it. The key aspect when using shared memory is to make sure

that a read from a memory block happens only after a write has occurred earlier. Also, two

processes should not simultaneously modify a memory block by writing at the same time.

It is the responsibility of the processes themselves to ensure synchronization occurs. When

multiple shared memory blocks are created to communicate with more than one process,

each shared memory block is identified by a unique key [19].

3.2.4 Mapped Memory

Mapped memory is similar to using files for IPC. In mapped memory, a link is created between

a file and the process’s memory space, thereby reducing the access time for file access. When

using files for IPC, memory access is in the filesystem where the file resides. But when using

mapped memory for IPC, a file is broken down into page-sized blocks and stored in the

process’s address space. These smaller blocks can be accessed without system calls resulting

in much faster communication between processes compared to filesystem access. Any block

that is created by one process can be accessed by another process as if it is accessing a

memory block in its own address space. [19].

3.2.5 Pipes

A pipe is a UNIX communication technique that permits unidirectional communication

between related processes. It is supported by UNIX-like and Windows operating systems.

All pipes have two end points, a read end and a write end. Data written to the write end


of the pipe is read back from the read end and is read from the pipe in the order in which

it is written. Pipes can communicate between two threads in a single process or between

parent and child processes. Writing data into a pipe and reading data from a pipe includes

flow control. If the speed of writing or reading is greater than the other, the corresponding

process is blocked. Pipes automatically provide flow control between two processes [17].

Pipes reside in the kernel and communication between two threads in a single process is

shown in Figure 3.2. When a pipe is created, it returns two file descriptors, one for reading

and the other for writing. In Figure 3.2 , fd[0] and fd[1] are the two file descriptors used

for writing and reading respectively.

fd[1] fd[0]

Process

Pipe

Process

Kernel

Figure 3.2: Pipe in a single process

In a shell, pipes are created using the ASCII vertical bar character |. For example, the

command ls | sort creates a pipe between the ls and sort commands. The shell creates

a pipe, connecting the standard output of the ls process to the standard input of the sort

process. The output of the ls process is sent as input to the sort process and the result of

this command is printed on the terminal.

When a pipe is created, two file descriptors are returned. These file descriptors cannot be

passed to unrelated processes. When pipes are used, generally forking is done to create a child

process, which is identical to the parent process. Thus, the two file descriptors are available


to both processes. The two processes mutually agree on the direction of communication and

close the end of the pipe they are not concerned with. Simple system calls are needed to

write data into and read data from the pipe. In Figure 3.3, the parent process forks and

creates a child process. The communication between the parent and child process proceeds

as follows. Initially, the parent process uses fd[1] for writing and the child process reads the

written data using fd[0]. The child process then writes data using fd[1] which the parent

process reads using fd[0]. The unused file descriptor during a read or write process is simply

closed so that the other process can use it. Hence, only one pipe is used for communicating

between two processes that are related and reside in the same system.

fd[1] fd[0]

Pipe

Forking

Parent

Child

Flow of

Data

fd[1] fd[0]

Process

Kernel

Figure 3.3: Pipe in a single process after forking


3.2.6 Named Pipe

A named pipe is a similar to the pipe concept in UNIX and is a pipe that has a name in the

file system. They are also known as First In, First Out (FIFOs) and allow two unrelated

processes to communicate with each other. In order to communicate by means of a named

pipe, processes have to open a named pipe either for writing or reading. Named pipes

provide one-way communication between two processes since a single pipe cannot be opened

for reading and writing simultaneously. In order to establish full-duplex communication,

two different named pipes are used, so that each named pipe provides flow of data in one

direction. Figure 3.4 shows an example of full-duplex communication using two named pipes.

Process1 writes to FIFO2 using writefd1 and process2 reads from it using readfd2. In

the same way, process 2 writes to FIFO1 using writefd2 and process 1 reads from it

using readfd1. The read and write file descriptors are unique and individual to both the

processes.

FIFOs are created programmatically using the mkfifo function. The first argument is the

path in which the FIFO is created and the second parameter specifies the permissions to

the FIFO, either read or write [17]. FIFO access is similar to an ordinary file access. They

can be opened either for reading or writing and are blocking by default which means that

if one process opens a FIFO for reading, the process will block until some other process

opens it for writing. This is true even if a process opens a FIFO for writing; writing will

be blocked until another process opens the FIFO for reading. FIFOs can also be opened in

non-blocking mode in case it is desired. But in the default mode, FIFOs are blocking and

provide automatic synchronization. Low-level I/O functions such as open, read, write,

close or C library functions such as fopen, fwrite, fread, fclose can be used to access

FIFOs.

When named pipes are used for full-duplex communication, care must be taken regarding

the order in which the pipes are opened. If the named pipes are opened incorrectly, deadlock

will occur. Deadlock is a situation in which processes wait indefinitely for each other to


Process

Kernel

Process1 Process2

FIFO2

FIFO1

writefd2 readfd2writefd1 readfd1

Figure 3.4: Two processes communicating with two FIFOs

complete. For example, consider there are two named pipes: PIPE1 and PIPE2 and two

processes: process1 and process2. In order to establish full-duplex communication, the

two pipes should be opened in the following order: if process1 opens PIPE1 for reading

and PIPE2 for writing then process2 must open PIPE1 for writing first and only then open

PIPE2 for reading. Deadlock occurs because if a FIFO is opened for reading it will block until

another process opens the same FIFO for writing. Swapping the order of the open calls will

result in both PIPE1 and PIPE2 opened for reading with no process opening any of the named

pipes for writing. Failure to observe the correct sequence will result in a deadlock situation

and processes waiting for each other to complete as shown in Figure 3.5(b). Figure 3.5(a)

shows the correct sequence of opening two named pipes for full-duplex communication.

3.2.7 Summary of IPC methods

The summary of the different IPC methods with their advantages and disadvantages are

shown in Table 3.1. The justification for choosing the IPC mechanism suitable for our


Process1 opens PIPE1

for reading


for writing


for writing


for reading


for reading


for writing


for reading


for writing

(a) No deadlock occurs (b) Deadlock occurs;

process1 and process2

wait indefinitely for

each other to complete

Figure 3.5: Deadlock in named pipes

experiment is given in Section 3.4

3.3 Sockets

The need for sockets arises from the fact that the IAP can reside in different systems across

a network. A socket is a bidirectional communication device which communicates in a point-

to-point fashion with a process running on the same machine or with a process running on a

different machine. It is one of the mechanisms by which two processes on different computers

can communicate and share data. Internet services and protocols such as World Wide Web,

FTP, and SMTP use sockets.

To create a socket, three parameters are required: communication style, namespace and


IPC Method Pros Cons

Files Easy access and portability Uses up disk space and access tends

to be slow

Queues cannot be reused and it is

Message Messages have priority and hence expensive for large volume of data because

queues can be read out-of-order the size of the queue is directly dependent

on the volume of data

Shared Fastest form of IPC available due Deadlock is a common event and synchronization

memory to its quick access to memory has to be done by the processes themselves

Mapped Simpler and efficient file access Failure to close the mapped memory segment by

memory within a single process the process will wait till system

shutdown to get rid of the segment

Pipes Simplest form of IPC and can be Only used for communication between

used inside a shell related processes

Can be used for communication between

Named unrelated processes and its access is It is a point-to-point communication technique

pipes similar to file access, hence modelling environment and cannot be used across networks

like MATLAB can access it as a file

Table 3.1: Advantages and disadvantages of different IPC methods

protocol. The communication style indicates the method by which the socket treats the

payload and specifies the number of communication devices. Data that is sent through a

socket is packaged into chunks called packets. A socket namespace indicates the way in

which socket addresses are written. Socket address can be in local namespace within the

same machine, or in Internet namespace which is the address of any machine connected to

the network with a port number. A port number is used to distinguish between multiple

sockets on the same host. Each host has 65,536 ports, of which some are reserved for specific

purposes. A protocol specifies how data is transmitted in the network. An example is

TCP/IP, which is used in the Internet and the UNIX local communication protocol [19].

3.3.1 Socket Types

The two main types of sockets are Transmission Control Protocol (TCP) and User Datagram

Protocol (UDP). A TCP socket stream guarantees delivery of all packets in the order they


were sent in. During a transmission, packets can be reordered or lost due to network errors.

To counter these errors, the receiver has the additional feature to request a retransmission

of the lost or reordered packet. In a TCP connection, the address of the receiver is known

beforehand and the sender establishes a connection with the receiver through handshake

messages. Only after this connection is created, data transfer occurs.

A UDP socket datagram does not guarantee delivery of all packets in the order they were sent

in. Packets can be lost or reordered during the transfer due to network errors. The receiver

does not account for these errors by requesting a retransmission. There are no handshake

signals between the sender and the receiver, hence the receiver’s address is sent along with

each packet when data transfer occurs.

The next sections explains different functionality associated with sockets.

3.3.2 Creating and Destroying Sockets

The socket and close functions create and destroy sockets, respectively. Creating a socket

requires three parameters: namespace, communication style, protocol. The namespace pa-

rameter consists of two values: AF LOCAL indicates communicating to a socket in the same

computer, and AF INET indicates communicating to a socket on a computer that can be

identified with a 32-bit IP address. AF stands for Address Families. The communication

style also consists of two values: SOCK STREAM specifies connection-style socket or TCP and

SOCK DGRAM specifies datagram-style socket or UDP. The protocol specifies the low-level

mechanism to transmit and receive data. It is generally given the value 0, which means it is

upto the service to decide the protocol it wants to use and is not decided by the user.

If socket call succeeds, a file descriptor is returned for the socket. As with other file opera-

tions, read and write can be done with this file descriptor. To remove the socket, the close

function is used [19].


3.3.3 Bind

The client must have the server’s address and port number so that it can send data to the

server. Similarly, the server must know the client’s address and port number so that it can

send data to the client. Binding is a process of assigning an address and port number to

the socket. The bind system call has three components: The first argument is the socket

file descriptor. The second argument is a pointer to the socket address structure; its format

depends on the socket’s address family. The third argument is the length of this address

structure, in bytes.

3.3.4 Sending Information

The methods used to write to or read from a file descriptor can be applied to a socket write

or read because sockets are identified with a file descriptor. The send or sendto functions

which are specific to sockets can be used as an alternative to the write system call. In the

same way, any technique that is used to read from a file descriptor is also applicable to a

socket. The recv or recvfrom functions are specific to sockets and can be used as alternative

to the read system call. Both the sendto and recvfrom calls have similar arguments and

need a socket descriptor, a buffer, length of the buffer and address of either the client or

server [19].

3.3.5 UDP Sockets

The connection-less socket is shown in Figure 3.6. In a typical computer network, the

server is the program which provides any service or resource and the client is the program

that requests the service or resource. UDP sockets do not create a connection before data

transmission and consists of a server and a client. The first step in both the processes is to

create a socket using the socket system call. After sockets are created successfully, a file

descriptor is returned corresponding to the socket. Both processes then call the bind system


call which is used to find the address of the process on the other end of the socket. The

client is now ready and sends data to the server using the sendto function and the server

receives this data using the recvfrom call. The recvfrom system call returns the number

of bytes it has received. The recvfrom system call in the server will block until it receives

data from the client. After the server receives data, it then sends data and the client receives

it using the same system calls as mentioned above. The final step is to remove the sockets

using the close system call. Design decisions to choose the type of socket are explained in

Section 3.4.

socket()

bind()

recvfrom()

socket()

bind()

sendto()

sendto() recvfrom()

Blocks until data is

received from the client

close()close()

Data

(request)

Data (reply)

Server Client

Figure 3.6: UDP socket connection


3.4 Design Decisions

This section deals with justification for choosing an IPC mechanism and socket type most

suited for our work. It also justifies use of structures to handle data in the experiment.

3.4.1 Named Pipes for IPC

Named pipes are chosen as the method of IPC. The reason for choosing named pipes over

other methods of IPC is because named pipes provide full-duplex communication between

unrelated processes as explained in Section 3.2.6. Also, named pipes are a built-in service

provided by the UNIX operating system, so no additional software packages are needed for

their support. It is supported on both the laptop and embedded platform used. Named pipes

provide automatic synchronization and hence reading and writing are guaranteed to function

correctly provided the pipes are opened in the correct sequence. They also have permissions

(read and write) associated with them unlike anonymous pipes. These permissions can be

used to enforce secure communication between two processes [20].

Another advantage named pipes have over other mechanisms of IPC is that named pipes

are identified by their access point, which is essentially a file that resides in the file system.

Because named pipes have the pathname of a file associated with them, it is possible for un-

related processes to communicate with each other. Hence, two unrelated processes can open

the file associated with the named pipe and begin communication as if they are interacting

with a file. Unlike anonymous pipes which are process persistent objects, named pipes are

file system persistent objects; that is, they exist beyond the life of the process. Once created,

named pipes always exist in the file system and any process can access them anytime as and

when desired. They have to be explicitly deleted by one of the processes or they have to be

deleted from a shell command line.

Named pipes are also supported by UNIX shells. Any control system would require a model-

ing environment to create an accurate model of the real physical system before it is deployed


in the real world. Named pipes are also treated as files in a modeling environment such as

MATLAB. MATLAB treats a named piped in the same way it treats a formatted data, text,

spreadsheet, or image file. No special functions are needed in MATLAB to open, access or

close named pipes.

3.4.2 UDP over TCP

TCP and UDP are transmission protocols which are used in socket programming, and allow

communication between processes on different systems. The decision to choose UDP over

TCP is primarily because of UDP’s lower latency [21]. The comparison of TCP and UDP

is shown in Table 3.2.

Parameter TCP UDP

Connection state TCP is a connection-oriented protocol UDP is a connection-less protocol

Connection Sets up connection using three handshake signals No connection setup is required

setup

Header size TCP header is 20 bytes UDP header is 8 bytes

Reliability Data arrives in the receiver in the order Data may or not arrive in order

in which it was sent

Flow control TCP has built-in flow control mechanism UDP does not provide flow control

Congestion Does not have a built-in congestion control

control Throttles the sender when the link becomes congested mechanism, but can be provided using

additional mechanisms

Higher latency since it guarantees delivery Lower latency since it does not maintain

Latency of packets in order and has built-in congestion connection state or have built-in mechanisms for

and flow control flow and congestion control

Is not suitable for real-time applications, Used in time-critical applications which can

Usage used mainly in applications where security and tolerate occasional packet loss, and suited to

reliability are more important than speed of data transfer control systems

Table 3.2: Comparison of TCP and UDP sockets

The reasons why UDP has lower latency compared to TCP are listed below:

1. No connection establishment : TCP uses a three-way handshake before it can begin data

transfer whereas UDP can transfer data without establishing a connection. This feature


of UDP is advantageous when processes exchange a continuous and low volume of data.

Control systems are a typical example where a low volume of data is transmitted every

control cycle.

2. No connection state: TCP maintains connection state in end systems. These con-

nection states consists of maintaining parameters such as send and receive buffers,

congestion control parameters, sequence and acknowledgement number parameters.

These parameters are responsible for implementing TCP’s reliable data transfer ser-

vice and providing congestion control. UDP does not track any of these parameters or

maintain connection state.

3. Small segment overhead : UDP has 8 bytes of header overhead in every segment com-

pared to TCP which has 20 bytes of overhead. The extra overhead in TCP is used to

guarantee reliability of service.

4. Unregulated send rate: TCP’s congestion control mechanism throttles the sender when

a link between a sender and receiver becomes congested. This throttling can have a

severe impact on real-time applications, which can tolerate some packet loss but are

more dependent on send rate. Transmission over UDP is constrained by the rate at

which application generates data and capabilities of the source like CPU, clock rate,

etc. UDP does not have a built-in congestion control mechanism to throttle the link

when congestion occurs. One of the ways by which congestion control is done in UDP

is by numbering the packets on the sender side and confirming their receipt in the

receiver. Any number that is missing will indicate a loss of a packet and the receiver

can inform the sender that a packet with that sequence number is missing.

5. Lower latency : In TCP, if a packet is lost the next packet will make it through; the

kernel will hold this data until the earlier packet is re-sent. This is because of the

guaranteed, in-order transmission of TCP and implies that new data will wait in the

kernel waiting for the TCP timeout so that the old data can be retransmitted. UDP

does not support this mechanism and is therefore preferred in low latency applications


like control systems.

3.4.3 Data and Memory Handling using Structures

The sections gives a brief overview of data and memory handling for our experiment. A

detailed explanation is provided in Chapter 4.

Control systems run in an infinite loop in order to control the plant. The controller gives

commands or instructions to the plant in the system to control and maintain a desired

state. The controller also generates data so that changes can be made to the behavior of the

plant. The current response of the plant decides which data is to be sent by the controller

in the next cycle and hence, the data being sent changes in every cycle of the loop. Plant

performance is altered by this data and they ultimately perform the desired function. The

data generated by the plant and that sent by the controller can be characterized as a list of

commands in the form of characters such as open check valve, increase flow rate, etc., or as

numerical values such as floating point numbers or integers. The data is converted into a

character stream in software irrespective of its data type and is then transferred among the

processes. The character stream thus converted is sent around the loop and is defined by

its length which is the number of characters that are transmitted. Whenever a data write

occurs, both the length and the data are written together; but whenever a read occurs, the

length is read first followed by the data. The number of characters that are read from the

data field depends on the length that was read previously. The length field is also converted

to a character stream.

Frame (256 bytes)

Data (0 – 255 bytes)Length (1 byte)

Figure 3.7: Frame used in the design


The length field and data field are identical in data type but differ in their sizes. The data

field is assumed to be from 0 to 255 bytes long and the length field is 1 byte long. Hence,

the maximum amount of data that can be transferred at any instant of time is 256 bytes

as shown in Figure 3.7. These two fields are combined into a single structure in software.

Structures are suitable for this type of transfer because it involves combining two different

fields which differ in their sizes. Hence, a single structure is declared which contains two

variables: the length field which is a single byte and the data field which is an array of up

to 255 bytes. For example, when data is being read from a named pipe or UDP socket, the

length field is read first and if the length is 4 then 4 bytes of data are read from the data

field.

This concludes the discussion on IPC mechanisms and design decisions to justify needs of

the experiment. The next chapter discusses the design in detail.

Chapter 4

Design Implementation

4.1 Incremental Design Versions

The control algorithm is a controller model implemented on a laptop, or controller code im-

plemented on an embedded platform: the ZedBoard. The control algorithm is implemented

either in MATLAB/Simulink on a laptop or is C-code generated from Simulink’s Embed-

ded Coder and implemented on an embedded platform. Since Embedded Coder generates

standalone C functions, there must be a process which invokes the generated C functions

periodically. The plant is also implemented in two different platforms, a plant model or

a real physical system implementing the process. The plant model is designed in MAT-

LAB/Simulink and is a replica of the real physical system. The other target is a real

physical system such as a Quanser SRV02 Rotary Inverted Pendulum (RIP) [22], a classic

and challenging control experiment. Interfaces are needed to help connect the controller and

the plant. Since there are two targets, there is a need for two different interfaces. The lap-

top and ZedBoard run Linux and are connected with a dedicated Gigabit Ethernet (GigE)

link. The SPI channels connect the ZedBoard to the RIP through an interface board. The

interface board is a custom interface board that avoids the use of Quanser’s standard Q2-

USB interface that does not suit an embedded host. The IAP supports seamless switching

35

Harish Balasubramanian Chapter 4. Design Implementation 36

between any of the four variations: MCMP, MCRP, RCMP and RCRP. The IAP must

perform this switching action without any run-time modifications or recompilation to the

control algorithm and through simple messages or commands.

Control system modeling and deployment proceeds with the following steps. Figure 4.1(a)

shows the initial approach consisting of an MCMP implemented in Simulink on a laptop.

There is no need for an IAP in this approach for control system design.

The next incremental step consists of a MCRP connecting the laptop to the RIP through the

ZedBoard and interface board. As shown in Figure 4.1(b), the ZedBoard has both GigE and

SPI channels to bridge the laptop and plant. Laptops normally do not support an embedded

protocol such as SPI.

Figure 4.1(c) illustrates the RCMP variation with the ZedBoard running controller code and

the plant modeled on the laptop. Simulink’s Embedded Coder generates C functions which

are run in a ZedBoard. The IAP allows the ZedBoard’s controller process to assume it is

communicating with the real plant using SPI, while it is actually communicating with the

laptop’s plant model using UDP sockets connections over GigE. Section 4.2 explains the

incremental steps in the RCMP design process.

The final version is an RCRP implementation with the controller code on the ZedBoard

connected to the RIP through an interface board. As shown in Figure 4.1(d), the controller

process connects the IAP to the SPI channels. The IAP optionally allows data logging on

the laptop which has a large and fast file system compared to the ZedBoard. The real-time

actuator output and sensor input data from the controller can be stored on a laptop file at

up to UDP GigE rates. Logged data can be reused in MATLAB/Simulink for additional

analysis and verification.


Laptop(Linux)

ZedBoard(Linux)

PhysicalPlant (RIP)

SPIGigE

(d) RCRP: Real Controller and Real Plant, with data logging on workstation

PROXY PROCESS

UDPSOCKETS

LOG CONTROLLERPROCESS

UDPSOCKETS

SERIALBUS DRIVERS

IAP

GigE

(c) RCMP: Real Controller and Modeled Plant

CONTROLLERPROCESS

IAP

UDP SOCKETS

PROXY PROCESS

UDPSOCKETS

SIMULINK

PLANT MODEL

SPIGigE

(b) MCRP: Modeled Controller and Real Plant

IAP

UDP SOCKETS

SERIAL BUS DRIVERS

PROXY PROCESS

UDPSOCKETS

SIMULINK

CONTROLLERMODEL

SIMULINK

(a) MCMP: Modeled Controller and Modeled Plant

PLANT MODEL

CONTROLLERMODEL

Figure 4.1: Incremental migration from fully-modeled to fully-implemented control system

4.2 RCMP Design Steps

The RCMP design was implemented in an incremental manner beginning with two simple

processes in C communicating over named pipes to the final implementation consisting of a

C process, two IAP processes and a MATLAB process communicating over named pipes and

UDP sockets. The design steps were done by adding a process to an existing and verified

design. The following are the steps in the RCMP design process.


4.2.1 Two C Processes Communicating with Named Pipes

The first step in the design process is discussed below and consists of two simple C processes.

Figure 4.2 shows the implementation with two C processes, one.c and two.c. The process

one.c first opens a named pipe, OneToTwo with write permissions and then another named

pipe, TwoToOne with read permissions. The process, two.c opens the same named pipes but

with permissions opposite to that of one.c. The process, one.c begins communication by

writing an integer to OneToTwo using the write command. Two.c receives the integer and

increments it by 1 and writes it back to TwoToOne. The received value is printed on the

terminal screen of one.c. The design is implemented in an infinite loop as control systems

are generally feedback systems running in an infinite loop. Named pipes are opened only

once in the beginning of the process and they can subsequently be used inside an infinite

loop.

The design was tested on the laptop and the ZedBoard. Initially, named pipes are created

using the mkfifo command. Two processes are executed in two different shells and run in

an infinite loop until stopped by the user.

Laptop

One.c

OneToTwo

TwoToOne Two.c

Figure 4.2: Two C processes communicating with named pipes


4.2.2 One C Process and One MATLAB Process Communicating

with Named Pipes

After two C processes communicate with named pipes, the next step in the design is to replace

one of the C processes with a MATLAB process. The C process one.c opens a named pipe,

CToMatlab with write permissions and MatlabToC with read permissions. The MATLAB

process two.m opens the same named pipes but with permissions opposite to that of one.c.

Figure 4.3 shows the implementation. The C process communicates to the MATLAB process

through CToMatlab and MATLAB communicates to the C process through the MatlabToC

pipe. The C process sends an integer to MATLAB and receives an incremented value from

MATLAB.

In MATLAB, named pipes are treated as the same way as files and are opened only once in

C and MATLAB and then used in the infinite loop. MATLAB is opened in a shell without a

grapical user interface and the two processes run infinitely. The design is only implemented

on the laptop and not on the ZedBoard as the plant is modeled in MATLAB and always

runs on the laptop.

Laptop

One.c

CToMatlab

MatlabToC Two.m

Figure 4.3: One C process and one MATLAB process communicating with named pipes


4.2.3 UDP Sockets - Server and Client

The next part of the design was focused on getting the laptop to communicate with the

ZedBoard. Sockets were chosen as the medium of communication and UDP was chosen

as the protocol. The programming language chosen was C because the ARM core on the

ZedBoard supports C and the sockets API is well supported in C. The design consists of

two C programs, server.c and client.c. The server provides a service or resource and the

client makes a request for the same.

The first step is to create sockets the client and server; followed by a bind step so that the

socket’s end address is known to both the programs. Then, the client.c sends an integer

to server.c which receives the integer, increments it and sends it back to the client.

Sockets were first implemented on the laptop with both the server and client residing in

the laptop. The two processes are executed in two separate shells and the IP address of

the server is passed to client. Since it is a type of localhost communication, the IP address

passed was 127.0.0.1. After sockets were tested on the laptop, the next step was to test the

communication between the laptop and ZedBoard and shown in Figure 4.4. The server

was executed on the laptop and the client was executed on the ZedBoard and is a type

of remote host communication where the systems are connected through a network cable.

The IP address of the laptop is passed in the ZedBoard and is set to 192.168.1.1 since the

ZedBoard has a static IP address of 192.168.1.10. The UDP sockets constitute the IAP in

the design and was tested to run in an infinite loop since control systems run for many cycles

or infinitely.

4.2.4 One C Process and IAP

The penultimate step in the design consists of testing a C process communicating with

named pipes to the IAP as shown in Figure 4.5. Since the IAP is also in C, named pipes are

supported in it. The design was implemented with Cprocess.c and client.c executed on


ZedBoard

Client

Laptop

ServerUDP sockets

Figure 4.4: Server and client communicating over UDP sockets

the ZedBoard and server.c implemented on the laptop. Cprocess.c opens the named pipes

CToClient and ClientToC with write and read permissions respectively. An IAP process

client.c opens the same named pipes but with permissions opposite to that of Cprocess.c.

The server and client communicate through UDP sockets as explained in Section 4.2.3.

The infinite loop begins with Cprocess.c sending an integer to the client implemented

on the ZedBoard and the client, in turn, passing the integer to the server implemented

on the laptop. The server increments the integer and sends it back to the client, which

passes it to Cprocess.c. Named pipes are open only once in the C process and IAP. The

IP address of the laptop, 192.168.1.1, is passed to the ZedBoard.

ZedBoard

Cprocess

CToClient

ClientToC Client

Laptop

ServerUDP sockets

Figure 4.5: One C process and IAP communicating with named pipes and UDP sockets


4.2.5 RCMP Implementation

The real controller and modeled plant was implemented on the laptop and ZedBoard. The

laptop processes was implemented on an 4-core 2.8 GHz Intel Core i5 processor and 4 GB of

RAM and the ZedBoard contains a 667 MHz Cortex-A9 ARM processor. Both the laptop

and ZedBoard implement two processes each. The laptop implements Mprocess.m and

an IAP process server.c. The ZedBoard implements Cprocess.c and an IAP process

client.c. The MATLAB process is an m-script file and the other processes are captured

in C. Compiling is done in the laptop. The processes which are implemented on the laptop

are compiled using gcc whereas the processes which are implemented on the ZedBoard are

cross-compiled on the laptop using the arm-xilinx-gcc compiler and transferred using the

scp protocol across the Ethernet cable.

The cycle is shown in Figure 4.6 and begins with the Cprocess.c which is implemented on

the ZedBoard and is the controller algorithm which controls the system and creates data to

be sent to the plant model for simulation. Cprocess.c opens two named pipes, CToClient

and ClientToC, with write and read permissions respectively. The first step is to write to the

pipe CToClient. The client also opens the same pipes as Cprocess.c but with permissions

opposite to that of the Cprocess.c. Additionally, the client also opens a socket connection

so that it can connect to the laptop IAP process. The client reads the data from CToClient

and writes it to the socket it created. The IP address of the laptop, 192.168.1.1 is passed to

the client process.

The laptop implements the IAP process server. The server also creates a socket to

connect to the client which runs on the ZedBoard. The server also opens two named

pipes, ServerToMatlab and MatlabToServer, with write and read permissions respectively.

Data is received in the server and is written to ServerToMatlab. The MATLAB script,

Mprocess.m, opens the same pipes as server but with permissions opposite to that of the

server. The Mprocess.m is the plant model, which is a replica of the physical system,

receives data from the server on ServerToMatlab. The plant model simulates the system


with the received data and writes to the server on the MatlabToServer pipe. MATLAB

can handle named pipes in the same way it handles files. The server reads the data from

the pipe and writes it to the socket it created. The IP address of the ZedBoard, 192.168.1.10,

is acquired by the server during the phase when the client writes to the server. The

client receives data on the socket and writes it to the ClientToC pipe. The final step

is for Cprocess.c to read the data from the ClientToC pipe. This completes one control

cycle where data is created by the control algorithm, Cprocess.c and is simulated by the

plant model, Mprocess.m and is received back in the Cprocess.c. The cycle occurs in an

infinite manner using named pipes for transmission within a host system and using sockets

for transmission between a host and a remote system.

ZedBoard

Cprocess

CToClient

ClientToC Client

Laptop

Server

ServerToMatlab

MatlabToServer MprocessUDP sockets

Figure 4.6: RCMP implementation

The manner in which data is handled in the RCMP implementation is shown in Table 4.1.

The previous chapter justified using structures for data handling. Cprocess.c declares a

struct frame which has two variables, unsigned char length and

unsigned char data[length]. In the same process both length and data are initialized.

For example, length is given the value 5 and data is given the value 12345. The data is an

integer in this example and is converted into a character stream using the sprintf function.

The entire frame is now sent to the client, which is also implemented on the ZedBoard. The

client reads the length field first, which is 5, and then reads 5 characters from the data field.

The client then sends these 6 characters to the server which then reads and writes these 6

characters to MATLAB. In MATLAB, length is read as an uint8. The data is read as an

unsigned char depending on the value of the length field which is converted back into an


integer using the sscanf function. Using the received data, it can make changes to the plant

behavior and performance. Data is sent back around the loop in the same manner. The

important aspect in the design is that Cprocess.c which implements the controller is aware

of the initial values of length and data and other processes are not aware of the data type

of the data or the length of the character stream. They simply read the length field first

followed by that many characters in the data field.

This concludes the design implementation along with low level design details. Experimental

results and analysis is done in Chapter 5.

Process Length Handling Data Handling

Cprocess.c Length is initialized Data is initialized and converted

to a character stream

Client.c Length is read first Data is read as a character stream

based on the length read initially

Server.c Length is read first Data is read as a character stream

based on the length read initially

Mprocess.m Length is read as an integer Data is read as a character stream

based on the length and converted to an integer

Table 4.1: RCMP data and memory handling

Chapter 5

Results

The parameters used to measure system performance are explained in Section 5.1. Sec-

tion 5.2 gives an overview of timestamps and lists the process timestamps without synchro-

nisation. Network Time Protocol and its role in synchronization is explained in Section 5.3.

The process of measuring delay and experimental assumptions are explained in Section 5.4.

Section 5.5 analyses the results obtained for the experiment

5.1 Parameters used to Measure System Performance

To measure the performance of any real-time system, two important parameters are consid-

ered: throughput and latency. Throughput is the rate at which data or commands are issued

and latency is the delay between an event and a response to the event [21]. In a closed-loop

control system which contains feedback, latency is a significant parameter of the system and

determines its proper functioning. If latency is not within an acceptable range in a control

system it can lead to system becoming unstable leading to unexpected behavior and severe

consequences.

The previous chapter explained the low-level design of the system with two processes im-

45

Harish Balasubramanian Chapter 5. Results 46

plemented on the ZedBoard and two process implemented on the laptop. Since there are

two different platforms on which the processes are implemented, measuring latency with the

current design setup, would result in incorrect values. The two different platforms consist of

different networks whose notion of time is very different, therefore measuring latency would

give inaccurate values. The next step is to demonstrate that the timestamp of an event

which occurs in quick succession across the networks is not in the same range and drifts with

time.

5.2 Obtaining Timestamps

In order to confirm that measuring latency across the different networks would result in

incorrect latency values, a simple experiment is conducted which involved recording times-

tamp. Any event is associated with a unique timestamp which indicates when the event

actually occurred in time [23]. The design consists of four processes: three implemented in

C and one in MATLAB. The MATLAB process, M-process.m was excluded from the design

and only the processes implemented in C were chosen because it easier to obtain timestamps

from a C process compared to a MATLAB process or from any simulation environment. The

experiment thus consisted of Cprocess.c and an IAP process client.c implemented on the

ZedBoard and another IAP process server.c implemented on the laptop. A packet was sent

across the network and the timestamp was recorded for every process in the network. The

timestamps were measured in the three different processes running in two different networks.

5.2.1 Function to Obtain Timestamp and Unix time

The function used to a get a timestamp in C is clock gettime defined in <time.h> and

consists of two parameters: clockID and timespec structure to measure time. Multiple

clocks are supported and implemented which are specified by the clockID. The two variables

which are part of the timespec structure, denote the time in seconds and nanoseconds.


The ideal clock for the experiment is the CLOCK REALTIME which returns the system-wide

clock and is measured as the number of seconds and nanoseconds elapsed since the Unix

Epoch. Before recording the timestamp of a process, an important step is to determine the

clock’s resolution using the clock getres function. Any measurement made will have the

amount of precision indicated by the resolution of a clock. To determine clock resolution

of CLOCK REALTIME on a laptop the clock getres function can be used which is supported

in a Linux environment. A simple program was implemented on the laptop to determine

resolution of CLOCK REALTIME and was found to be 1 nanosecond. The Unix Epoch is also

known as Unix time or POSIX time and returns the number of seconds passed since 00:00:00

Coordinated Universal Time (UTC), Thursday, 1 January 1970 [24]. The laptop used in

the design has the same notion of time, that is, any system time it measures is from this

Epoch. The Epoch is constant and acts as a reference clock and does not change even if the

laptop is powered down or restarted. The ZedBoard on the other hand is not synchronized

with this time Epoch and its time is reset every time a power cycle is done on it. Whenever

the ZedBoard is powered down and up again, the reference clock on the ZedBoard is reset.

Since the clock is reset with a power cycle, there is no fixed time reference to measure system

time. Clock synchronization and resolution on the ZedBoard is explained in Section 5.3.2

5.2.2 Timestamp of the Processes without Synchronisation

Table 5.1 and Table 5.2 below shows two iterations of the timestamp measured for a packet

travelling across the network from Cprocess.c to server.c. Each timestamp measurement

consists of time measured in seconds and nanoseconds. The packet has two timestamps

recorded on the ZedBoard: first in Cprocess.c and the second in client.c. The packet then

moves across the network and a timestamp is measured in server.c on the laptop. Figure 5.1

shows the movement of a packet across a network and the processes where timestamps of

the packet are recorded. The process of recording the timestamps is done multiple times

to ensure correctness and verify it. The results confirm that on the laptop, the elapsed


time is with respect to the start of the time Epoch but the elapsed time on the ZedBoard

is with respect to the last time the ZedBoard was powered ON. Since the networks have

different time sources as their reference clock, latency measurement will lead to incorrect

values. Therefore, before measuring latency in the network, ensuring that both the networks

use the same time reference is a necessary and important first step and is discussed in the

Section 5.3.

Process Time (sec)

Cprocess 676.968

Client 676.969

Server 1387383315.671

Table 5.1: Timestamp of the processes before synchronisation - iteration 1

Process Time (sec)

Cprocess 1438.969

Client 1438.971

Server 1387384077.681

Table 5.2: Timestamp of the processes without synchronisation - iteration 2

5.3 Network Time Protocol

The section deals with the Network Time Protocol (NTP) to synchronize clocks in different

devices across a network. The issue of drift is also discussed and the need for NTP is justified.

5.3.1 Need for NTP and Drift

Section 5.2 discussed the problems associated with measuring latency across a network in

which each sub-network had their own reference clocks. The problem is magnified when


ZedBoard

Cprocess CToClient Client

Laptop

ServerUDP sockets

Data packet Data packet Data packet

Figure 5.1: Processes where timestamp is recorded

drift of the clocks is taken into consideration. Drift can occur in clocks even if they are

set accurately initially and occurs when a clock runs at a different frequency compared to a

standard reference clock. For example, if the clock in a computer runs at a slower frequency

compared to a standard clock, then the computer’s clock will display a time which would

have already elapsed in real world time. The problem is amplified in a network which has

different clock sources as a reference clock source [25].

The laptop and ZedBoard, in addition to having different clock sources, also have drift in

their clocks. The ZedBoard’s on-chip clock runs at a significantly lower frequency compared

to the laptop. A simple experiment is conducted to verify that the ZedBoard clock has a

slower frequency. Terminals are opened on the laptop and ZedBoard and on it, the date

command is executed. The output for the command is displayed on the terminal and a after

about a minute (in real-world clock), the test is repeated. Output is compared for the two

different executions of the date command. The comparison confirms that the laptop clock

does not show any drift but the ZedBoard clock shows considerable drift and it is significantly

slower compared to the real-world clock. The drift is approximately two seconds per minute.


5.3.2 Clock Synchronization using NTP

The solution to overcome the issue of the clocks having different reference sources is by using

an existing network protocol, Network Time Protocol (NTP) [26]. NTP is used to achieve

clock synchronization for systems across different networks and its main goal is to ensure

that all systems connected in a network are synchronized to the Coordinated Universal Time

(UTC) [27].

NTP treats the clocks as different levels and to synchronize clocks, it uses a hierarchical

system in which the clocks at the lower level synchronize to the clocks at the higher level

[28]. The first level is called Stratum 0 and consists of high precision timers and are used as

reference clocks for the lower levels. Stratum 1 contains systems that are directly connected

to these reference clocks and are known as primary time servers. The systems in Stratum 2

get their clock reference from Stratum 1 and so on. Systems that reside in the same level can

synchronize among each other and this is similar to a peer-to-peer connection and systems

that reside in different levels are similar to a client-server connection model. An example of

this is shown in Figure 5.2.

Stratum 0 contains high accuracy timers that act as reference for other clocks in the lower

levels. They are also called as reference clocks. Each system in the lower level is a device and

queries other devices in the same or lower level. In Stratum 1, devices A and B synchronize

to the reference clock and also with one another to verify time synchronization. Devices P

and R in Stratum 2 connect to devices A and B respectively and the connection is similar

to a client-server connection model. Devices can not only synchronize to devices in higher

levels but also to devices in the same level. An example of peer-to-peer connection is device

Q which synchronizes to device P to setup its clock. Also, a single device can synchronize

two different devices in the lower levels such as device W and X which synchronize to device

P at the same time [29]. In Stratum 3, device Y creates two peer-to-peer connections with

devices X and Z and is done to verify the time synchronized from one device with another.

There are stratums ranging from 0 to 15 and stratum 16 is used to indicate that the clock


Reference Clocks Stratum 0

Client A Client B

Client P Client Q Client R

Stratum 1

Stratum 2

Client W Client X Client Y Client Z Stratum 3

Figure 5.2: Hierarchical levels of clocks

failed to synchronize with any other device. Each device runs an algorithm to decide whether

to create a client-server or peer-to-peer connection.

NTP is installed on the laptop which runs Linux OS and is built-in to the ZedBoard. The

clock on the laptop is synchronized to higher level clocks via the client-server model. The

clock on the ZedBoard is synchronized to the clock on the laptop through a peer-to-peer

connection model via the GigE cable.

Figure 5.3 shows the clock synchronization in the laptop and ZedBoard. The laptop is

connected to the global time servers across various levels through a client-server connection

model. The laptop sends out a confirmation message after its clock has been synchronized to

the devices in higher levels. The laptop resides in Stratum 7 of the clock hierarchy. After the

clock is synchronized on the laptop, the ZedBoard simply creates a peer-to-peer connection

via Gigabit Ethernet cable and connects to the laptop to synchronize its clock. Thus, we

have both kind client-server and peer-to-peer connection models in our network for time


Laptop ZedBoard

Global Time Servers

Client-Server Connection

Peer-to-Peer Connection

Figure 5.3: Synchronization setup

synchronization.

The first step is to ensure that the clock on the laptop is synchronized to the clocks in

stratum 0, which are the reference clocks. In the next step, the clock on the ZedBoard is

synchronized to the clock on the laptop by executing ntpd -q 192.168.1.1, which is the IP

address of the laptop. After a few seconds, the clock on the ZedBoard is synchronized to the

clock on the laptop which in turn is synchronized to the reference clocks. In order to verify

that the clocks are synchronized, the test which demonstrated that the clocks were not in

synchronization is repeated and timestamps are obtained from an empty packet which is sent

around the network. The test is repeated multiple times in order to ensure repeatability and

correctness. After clock synchronization has been achieved in the ZedBoard, an important

step is to measure the clock resolution on the ZedBoard using the clock getres function.

The major advantage of using NTP for clock synchronization is that it also adjusts clock

frequency of the clock on the ZedBoard [30]. Experiments to ensure clock synchronization

on the ZedBoard are short in duration which ensures clocks do not drift much during the test.

Even though clock frequency on the ZedBoard is 667 MHz, since NTP is used to achieve

clock synchronization the resolution of the clock is adjusted to 1 nanosecond. The same

program which was used to determine clock resolution in the laptop is used to determine


clock resolution on the ZedBoard. Thus, the timing apparatus on both the laptop and

ZedBoard have a clock resolution of 1 nanosecond which results in high-resolution timers for

latency measurement.

5.3.3 Timestamp of the Processes with Synchronisation

The section deals demonstrates that the clocks of the ZedBoard and laptop are synchronized.

Table 5.3 and Table 5.4 shows the timestamps of the three processes after synchronization.

Process Time (sec)

Cprocess 1387385575.589

Client 1387385575.589

Server 1387385575.590

Table 5.3: Timestamp of the processes after synchronisation - iteration 1

Process Time (sec)

Cprocess 1387385987.391

Client 1387385987.391

Server 1387385987.392

Table 5.4: Timestamp of the processes after synchronisation - iteration 2

The results show that the timestamps obtained in the Cprocess.c, client.c and server.c

are in the same range as one another and the two systems in the network have the same

clock for reference, thus ensuring clock synchronisation. After the clocks are synchronized, it

is safe to measure latency since the primary reason which results in incorrect latency values

has been eliminated. Clock drift can still affect the latency measurement and its effect has

to be minimized or negated. In order to minimise the effect of clock drift on the latency

measurement, the command to synchronize the clocks is first executed before every run of the

experiment to measure latency [31]. This is done by executing the ntpd -q 192.168.1.1 on


the ZedBoard every single time before measuring the latency. After the successful completion

of the ntpd command, the date command is executed on both the laptop and ZedBoard.

The output of the date command on the laptop and ZedBoard are compared and if the

outputs are in the same range of each other, it confirms that the drift is minimal between

the clocks of the two systems. The date command is executed on the two systems in quick

succession to mitigate external factors affecting the measurement. These external factors

are mainly human inputs on the two terminals to execute the date command and care must

be taken to execute the command in quick succession or as close together as possible. The

next section shows how latency is measured and lists the experimental assumptions during

latency measurement.

5.4 Delay Measurement

A critical parameter in any control system is the delay. Delays in control systems are due

to initialization, computation and communication. Communication delays add significant

amount of latency in any control system since it involves different components implemented

on different platforms. Computation delays depend on the implementation platform.

Figure 5.4 represents the system which is setup with a laptop and a ZedBoard connected

via a GigE cable, creating a closed network. There are two processes implemented on both

the laptop and ZedBoard. Cprocess.c and client.c are executed on the ZedBoard and

server.c and Mprocess.m are executed on the laptop. A loop around the system begins

with the Cprocess.c providing an input or command signal to the client.c. The signal is

transferred across the GigE to the server.c which in turn feeds the Mprocess.m to complete

the loop. The loop continues in the next iteration with the output from the Mprocess.m

transmitting to the server.c and eventually the Cprocess.c.


CprocessCToClient

ClientToC

Client ServerServerToMatlab

MprocessUDP sockets

ServerClientUDP sockets MatlabToServer

Figure 5.4: Control cycle loop with IAP

5.4.1 Experimental Assumptions when Measuring Delay

The following assumptions and steps are made before measuring delay in a control loop.

1. The clocks of both the laptop and ZedBoard are synchronized and is accomplished using

the Network Time Protocol, which synchronizes the clock on the ZedBoard using the

clock on the laptop as a reference. The laptop and ZedBoard create a peer-to-peer

network via the GigE cable.

2. Drift is minimized between the clocks. The phenomena of drift causes the clocks which

operate at different frequencies to lose notion of time. In order to minimize drift, the

command to synchronize the clocks is executed before every single iteration of latency

measurement which minimizes drift appreciably.

3. The accuracy of delay is affected by the number of processes that are executed at any

given time on the systems. The ZedBoard has a small processor and its performance

is significantly affected when a lot of processes are implemented on it. The extraneous

processes on the ZedBoard are shut down or closed before measuring delay. Even

though the processor on the laptop is more powerful compared to that of the ZedBoard,

any irrelevant and extraneous processes are terminated.

4. When measuring delay, there is no data or message sent to the stdout in the processes.

By eliminating printing on the terminal, it is ensured that printing messages to stdout


does not become the bottleneck when measuring delay. It is observed that printing to

stdout is a bottleneck since the control cycle halts in the middle of an execution cycle

because the output buffer for stdout fills up and requires user input to clear it. After

the buffer is cleared, execution continues normally but stopping the control cycle is

undesirable.

5. Delay is measured in the control cycle after the 1000’th iteration to make sure that

initialization overheads are eliminated and caching becomes effective. After the 1000’th

iteration, control cycle delay is measured within one control cycle since the system

becomes stabilized with increase in control cycles.

Sections 5.4.2 and 5.4.3 explains the components of delay within one control cycle before

and after the addition of the IAP. Results are summarized and analyzed in Section 5.5.

5.4.2 Measuring Delay in the Loop without IAP

When an IAP is not present, a control cycle consists of a controller process, Cprocess.c

and a plant process, Mprocess.m implemented in MATLAB. The setup is implemented on

the laptop only since there are no additional interfaces needed for the two processes to

interact. Delay in the control cycle consists of a controller and plant computation delay and

a communication delay between the plant process and controller process. It is much harder

to obtain timestamps from a modeling environment compared to C and therefore timestamps

are calculated in Cprocess.c only. The different components of the delay are listed below.

1. The first component of the delay is the computation time of the controller process.

To measure delay, a timer is run before the process starts and another timer is run

after the process has completed. The computation time in a controller process is the

difference between the two timer values.

2. Communication time between the controller and plant process is the next component

of delay. A timer is initialized before the communication begins with the plant process


and stopped after the communication has been received from the plant process since

timestamps are obtained in C only. The difference in timer values indicates the two-way

communication time between a controller and plant process.

3. The final component of the delay is the computation time of the plant process. The

plant process is implemented in MATLAB and timestamps are not accurate when

measured in a modeling environment. Therefore, the total latency in the control cycle

is measured and delays measured in Steps 1 and 2 are subtracted. The remaining delay

is an approximate value of computation delay in a plant process.

Table 5.5 shows the mean and standard deviation of the different components of the delay

over fifty iterations of the same experiment. Delay is measured in the control cycle after 1000

iterations to ensure system is stabilized and repeated multiple times to ensure correctness

and consistency of the result.

5.4.3 Measuring Delay in the Loop with IAP

After the addition of the IAP, a control cycle consists of a controller process, Cprocess.c

and an IAP process,client.c implemented on the ZedBoard and an IAP process, server.c

and plant process, Mprocess.m implemented on the laptop. The ZedBoard and laptop are

connected via GigE cable. Delay in the control cycle consists of a controller and plant

computation delay and a communication delay between the controller and plant with the

IAP and an IAP to IAP communication delay since the two IAPs are implemented on

different platforms. The delay measurement consists of obtaining timestamps from processes

implemented in C only and not from the modeling environment. Different components

contributing to the delay are listed below.

1. Computation delay in the controller is the first component of the delay. The compu-

tation time is measured by subtracting timer values before and after the process has

started and completed.


2. The second component of the delay is the time involved in communication between

the controller process and an IAP process, client.c. A timestamp is obtained at the

beginning of a write in Cprocess.c and another timestamp is obtained at the end

of a read in client.c. Both these processes are implemented on the ZedBoard and

the delay is obtained by simply subtracting the two timestamps. The same process is

repeated for measuring delay from client.c to Cprocess.c.

3. The next component of the delay is the time for the two IAP processes to commu-

nicate. To measure delay, a timestamp was obtained at the start of data transfer in

client.c and another timestamp was obtained when data was received in client.c.

The difference in the two timestamps results in the two-way communication delay from

client.c to server.c.

4. The fourth component of the delay is time taken for an IAP process, server.c to com-

municate with the plant implemented in MATLAB. A timer is initialized in server.c

before the communication begins with the plant process and stopped after communi-

cation has been received from the plant process. The difference in timer values shows

the two-way communication time between an IAP and plant process.

5. The final component of the delay is the computation time of the plant process. To

determine this, the total control cycle latency is measured for one loop and the delays

calculated in Steps 1, 2, 3 and 4 are subtracted. The result is an approximate value of

computation time in the plant.

Delays are measured in the control cycle loop after 1000 iterations of the control cycle and the

experiment is repeated multiple times to determine delay average and ensure repeatability

and consistency of the experiment. Table 5.6 shows the mean and standard deviation of the

different components of the delay for fifty iterations of the same experiment.


5.5 Results and Analysis

Table 5.5 and Table 5.6 represent the mean and standard deviation of the components

of delay measured for one control cycle before and after adding the IAP to the design,

respectively. The measurements are taken after 1000 control cycles when the system has

stabilized and initialization overheads are minimized.

Component Mean (ms) Standard Deviation (ms)

Controller 0.019 0.002

Computation

Plant 0.035 .009

Computation

Controller to

Plant 2.233 0.161

Communication

Total Delay 2.291 0.157

Table 5.5: Mean and standard deviation of delay components without IAP over 50 iterations

5.5.1 Computation Delay with and without IAP

Figure 5.5 shows the computation delay in the controller before and after the addition of

an IAP over fifty iterations of the experiment. The controller is implemented on the laptop

when an IAP is not present and on the ZedBoard when an IAP is present. The average

computation delay is 0.074 milliseconds in the controller after the addition of the IAP. The

processor on the laptop is much faster than the processor on the ZedBoard which results in

the delay on the ZedBoard.

Plant computation delay is represented in Figure 5.6 before and after the addition of an IAP.

The plant process is always implemented on the laptop and the plant computation delay is


Component Mean (ms) Standard Deviation (ms)

Controller 0.074 0.003

Computation

Plant 0.034 .007

Computation

Controller to IAP 1.734 0.107

Communication

IAP to Plant 1.415 0.072

Communication

IAP to IAP 0.206 0.029

Communication

Total Delay 3.465 0.132

Table 5.6: Mean and standard deviation of delay components with IAP over 50 iterations

not affected by the IAP. The delay is measured to be about 0.035 milliseconds.

5.5.2 Communication Delay with and without IAP

Figure 5.7 shows the communication delay before and after the addition of an IAP. The com-

munication delay before the addition of the IAP is 2.233 milliseconds and consists of only

one delay component. The inclusion of the IAP creates three delay components which in-

creases the communication delay significantly and the total communication delay is averaged

to be 3.355 milliseconds. The communication between the IAP and the plant or controller

contributes the major component of this delay and the IAP to IAP communication is very

small and is less than 0.25 milliseconds. Another interesting feature is that the standard

deviation of the individual components of the delay are small and all values lie close to each

other. Communication delay is significantly higher compared to computation delay in con-

trol systems and reiterates the fact that communication can become a bottleneck in control


0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

De

lay

(in

mill

ise

con

ds)

Iteration number

Computation Delay of Controller with and without IAP

Controller computation without IAP Controller computation with IAP

Figure 5.5: Computation delay of controller with and without IAP

systems. Computation delays are trivial compared to the communication delay.

5.5.3 Control Cycle Delay with and without the IAP

Figure 5.8 shows the total delay in one control cycle with and without the IAP for fifty

iterations of the experiment. The average total delay without the IAP is measured to be

2.291 milliseconds with a standard deviation of 0.157 milliseconds. The average total delay

after the addition of the IAP is calculated to be 3.465 milliseconds and a standard deviation

of 0.132 milliseconds. An important feature is the consistency of the measured delay values.

No abnormal or inconsistent values were observed, demonstrating that the experiment is

repeatable and measured delay values are consistent. Figure 5.9 represents the percentage

increase in overall delay after the addition of the IAP. The overall delay has increased by


0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Dela

y (in

mill

isec

onds

)

Iteration number

Computation Delay of Plant with and without IAP

Plant computation without IAP Plant computation with IAP

Figure 5.6: Computation delay of plant with and without IAP

about 55-60% but the total delay after the addition of the IAP still lies within a few millisec-

onds. The small delay value allows control systems to operate efficiently without becoming

unstable.


0

0.5

1

1.5

2

2.5

3

3.5

4

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Dela

y (in

mill

isec

onds

)

Iteration number

Communication Delay with and without IAP

Communication without IAP Communication with IAP

Figure 5.7: Communication delay with and without IAP

00.5

11.5

22.5

33.5

4

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Dela

y (in

mill

isec

onds

)

Iteration number

Total Delay in One Control Cycle with and without IAP

Total time without IAP Total time with IAP

Figure 5.8: Total delay in one control cycle loop with and without IAP


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Perc

enta

ge

Iteration number

Percent Increase in Delay with and without IAP

Total time without IAP Total time with IAP

Figure 5.9: Percent increase in total delay in one control cycle loop after addition of an IAP

Chapter 6

Conclusions

6.1 Motivation

Control systems exist everywhere in the world around us. Embedded platforms are being

used in control system design since they are powerful, simple and fast. The traditional design

steps involved in deploying control systems are complex and involve migrating directly from

a modeled plant and modeled controller to a real plant and real controller.

Supporting intermediate design versions helps in early verification and validation of control

systems which requires the interaction between an embedded platform and control system to

be fast, secure and flow controlled. The effects of faulty communication are well documented

and result in unstable control systems which can have a disastrous impact. Unstable control

systems can cause loss of human life as shown in the Chernobyl nuclear disaster and SAAB

JAS-39 aircraft accident [32].

65

Harish Balasubramanian Chapter 6. Conclusions 66

6.2 Contributions

This work describes the addition of an extra process, an IAP, to traditional industrial control

system design. The implementation of the IAP is simple and straightforward and abstracts

the underlying communication interface. ICS design proceeds in incremental steps from a

modeling to real-world environment. The IAP supports incremental design steps by support-

ing intermediate versions without increasing design complexity. The addition of the IAP is

inconspicuous in traditional control system design and does not severely affect latency in the

system.

A real controller is implemented on the Zynq-7000 All Programmable SoC and a plant is

modeled on MATLAB/Simulink. Named pipes serve as communication mechanism within

the plant or controller. Sockets are used for interaction between the plant and controller.

Sockets and named pipes provide automatic flow control and prevents overwhelming of plant

or controller. IAP is implemented in C and is independent of application code. The in-

cremental design version supported is RCMP and there are two additional IAP processes.

Latency in one control cycle is measured before and after the addition of the IAP. Individual

delays for each computation and communication are measured and analyzed. Experiments

to measure latency are repeatable and consistent. However, the latency after the addition

of an IAP increases appreciably and hence the IAP is removed before control systems are

deployed.

6.3 Future Work and Recommendations

The following are some recommendations for future work:

1. Implementation and verification of the IAP in a real-world environment. The design

could be tested with a RIP and verify that control action is unaffected by the addition

of the IAP.

Harish Balasubramanian Chapter 6. Conclusions 67

2. Measure timestamps in a simulation enviroment for improved accuracy in latency mea-

surement.

3. Incorporate a security layer to the design and measure how it affects latency so that

the entire system becomes more secure.

4. Improve functionality of the IAP to facilitate rapid switching between different inter-

mediate design versions. Little overhead is required since switching must be rapid.

Bibliography

[1] S. Bhattacharya and B. K, Control Systems Engineering. Dorling Kindersley, 2009.

[2] M. Jelali, “An overview of control performance assessment technology and industrial

applications,” Control Engineering Practice, vol. 14, no. 5, pp. 441 – 466, 2006.

[3] D. E. Seborg, T. F. Edgar, D. A. Mellichamp, and F. J. D. III, Process Dynamics and

Control. Wiley, 2011.

[4] W. Bolton, Programmable Logic Controllers (Fourth Edition). Oxford: Newnes, fourth

edition ed., 2006.

[5] B. Galloway and G. Hancke, “Introduction to industrial control networks,” Communi-

cations Surveys Tutorials, IEEE, vol. 15, pp. 860–880, Second 2013.

[6] T. Barlas, Developing FPGA-Based Embedded Controllers Using Matlab. INTECH

Open Access Publisher, 2010.

[7] MathWorks, “Control tutorials for MATLAB and SIMULINK,” 2012.

[8] C. Elliott, V. Vijayakumar, W. Zink, and R. Hansen, “National Instruments LabVIEW:

A Programming Environment for Laboratory Automation and Measurement,” Journal

of The Association for Laboratory Automation, vol. 12, pp. 17–24, 2007.

[9] Altera Corporation, “DSP Builder Handbook volume 1: Introduction to DSP Builder,”

May 2013. http://www.altera.com/literature/technology/dsp/hb_dspb.pdf.

68

Harish Balasubramanian Bibliography 69

[10] MathWorks, “Real-Time Workshop Embedded Coder,” October , 2004. http://www.

mathworks.com/help/releases/R13sp2/pdf_doc/ecoder/ecoder_ug.pdf.

[11] MathWorks, “Control systems,” Oct 22, 2012.

[12] T. Lennon, “Model-based design for Mechatronics Systems,” Machine Design, Novem-

ber 2007.

[13] Xilinx, Inc, Zynq-7000 All Programmable SoC Overview, v1.6 ed., Dec

2, 2013. http://www.xilinx.com/support/documentation/data_sheets/

ds190-Zynq-7000-Overview.pdf.

[14] Avnet, “ZedBoard,” 2012.

[15] MATLAB, version 8.0.0.783 (R2012b). Natick, Massachusetts: The MathWorks Inc.,

2012.

[16] M. Khambatti, “Named Pipes, Sockets, and other IPC,” 2001.

[17] W. Stevens, Unix Network Programming. UNIX Network Programming Series, Prentice

Hall PTR, 1998.

[18] D. Rusling, The Linux Kernel. Pearson P T R, 2000.

[19] M. Mitchell and A. Samuel, Advanced Linux Programming. Thousand Oaks, CA, USA:

New Riders Publishing, 2001.

[20] S. Pate and F. V. D. Bosch, UNIX Filesystems: Evolution, Design and Impemenation.

New York, NY, USA: John Wiley & Sons, Inc., 2003.

[21] J. F. Kurose and K. W. Ross, Computer Networking: A Top-Down Approach. Addison-

Wesley Longman Publishing Co, Inc., fifth ed., March 2009.

[22] Quanser, “The Rotary Control Lab,” 2012.

[23] M. Kerrisk, The Linux Programming Interface: A Linux and UNIX System Program-

ming Handbook. San Francisco, CA, USA: No Starch Press, 1st ed., 2010.

Harish Balasubramanian Bibliography 70

[24] T. Christiansen and N. Torkington, Perl Cookbook. Sebastopol, CA, USA: O’Reilly &

Associates, Inc., 1998.

[25] D. L. Mills, Computer Network Time Synchronization: The Network Time Protocol.

Boca Raton, FL, USA: CRC Press, Inc., 2006.

[26] D. Waldron, “What is network time protocol (NTP)? a definition and 5 reasons

why your computer network needs it,” 2014. http://www.galsys.co.uk/news/

network-time-protocol-ntp-definition-5-reasons-computer-network-needs/.

[27] S.-M. Jun, D.-H. Yu, Y.-H. Kim, and S.-Y. Seong, “A time synchronization method

for NTP,” in Real-Time Computing Systems and Applications, 1999. RTCSA ’99. Sixth

International Conference on, pp. 466–473, 1999.

[28] D. Waldron, “What is NTP? what are its benefits?

find out now,” 2014. http://www.galsys.co.uk/news/

what-is-ntp-what-are-its-benefits-a-galleon-systems-guide/.

[29] T. Nakashima, S. Oshima, and A. Nakashima, “Implementation of the performance

evaluation system for the NTP server,” in Communications, Computers and signal Pro-

cessing, 2003. PACRIM. 2003 IEEE Pacific Rim Conference on, vol. 2, pp. 828–831

vol.2, Aug 2003.

[30] StackExchange, 2008. http://stackoverflow.com/questions/97853/

whats-the-best-way-to-synchronize-times-to-millisecond-accuracy-and-precision-b,.

[31] StackExchange, 2014. http://unix.stackexchange.com/questions/118631/

how-can-i-measure-and-prevent-clock-drift,.

[32] G. Stein, “Respect the unstable,” Control Systems, IEEE, vol. 23, pp. 12–25, Aug 2003.

Appendix A

Code used in the Design - without

IAP

// F i l e − Cprocess . c

#inc lude <s t d i o . h>

#inc lude < f c n t l . h>

#inc lude <s t r i n g . h>

#inc lude <uni s td . h>

#inc lude <time . h>

#inc lude <s t d l i b . h>

#d e f i n e DEBUG 1

#d e f i n e MAX BUF LENGTH 5

#d e f i n e PIPE OUT ”CToMat”

#d e f i n e PIPE IN ”MatToC”

s t r u c t frame {

unsigned char l ength ;

unsigned char buf [MAX BUF LENGTH] ;

71

Harish Balasubramanian Appendix 72

} ;

i n t main ( )

{

s t r u c t t imespec s ta r t , l a s t , startcomp , endcomp , startcomm , endcomm ;

double t1 , t2 , d i f f e r e n c e , t3 , t4 , t5 , t6 , t7 , t8 , t9 , t10 ;

s t r u c t frame send , recv ;

i n t p ipe out , p i p e i n ;

i n t count = 0 ;

unsigned i n t data send = 0 ;

unsigned i n t r e c v l e n g t h = 0 ;

unsigned i n t r ecv data = 0 ;

i n t i ;

p ipe out = open (PIPE OUT,OWRONLY) ;

i f ( p ipe out == 0)

{

f p r i n t f ( s tde r r , ” Cprocess could not open %s ” ,PIPE OUT ) ;

e x i t (−1);

}

p i p e i n = open ( PIPE IN ,O RDONLY) ;

i f ( p i p e i n == 0)

{

f p r i n t f ( s tde r r , ” Cprocess could not open %s ” , PIPE IN ) ;

e x i t (−1);

}

f o r ( i =0; i<MAX BUF LENGTH; i++)

{

send . buf [ i ] = 0 ;


recv . buf [ i ] = 0 ;

}

c l o c k g e t t i m e (CLOCK REALTIME, &s t a r t ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &startcomp ) ;

data send = 100 ;

data send = data send + 1 ;

f o r ( count =0; count <1005; count++)

{

s p r i n t f ( send . buf ,”%d” , data send ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &endcomp ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &startcomm ) ;

wr i t e ( p ipe out ,&send , (MAX BUF LENGTH)∗ s i z e o f ( char ) ) ;

read ( p ipe in ,& recv ,(4+1)∗ s i z e o f ( char ) ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &endcomm ) ;

s s c a n f ( recv . buf ,”%d”,& recv data ) ;

data send = recv data ;


}

c l o c k g e t t i m e (CLOCK REALTIME, &l a s t ) ;

t1 = ( ( double ) startcomp . t v s e c + 1 .0 e−9∗startcomp . tv ns e c ) ;

t2 = ( ( double ) endcomp . t v s e c + 1 .0 e−9∗endcomp . tv ns e c ) ;

t3 = ( ( double ) startcomm . t v s e c + 1 .0 e−9∗startcomm . tv nse c ) ;

t4 = ( ( double )endcomm . t v s e c + 1 .0 e−9∗endcomm . tv nse c ) ;


t5 = ( ( double ) s t a r t . t v s e c + 1 .0 e−9∗ s t a r t . t v n s e c ) ;

t6 = ( ( double ) l a s t . t v s e c + 1 .0 e−9∗ l a s t . t v n s e c ) ;

p r i n t f (” Computation time o f Cprocess = %.7 f \n” , t2−t1 ) ;

p r i n t f (” Communication time = %.7 f \n” , t4−t3 ) ;

t7 = t2−t1 ;

t8 = t4−t3 ;

t9 = t6−t5 ;

t10 = t9 − ( t7+ t8 ) ;

p r i n t f (” Matlab computation time = %.7 f \n” , t10 ) ;

p r i n t f (” Total time = %.7 f \n” , t6−t5 ) ;

r e turn 0 ;

}

%F i l e − Mprocess .m

p ipe out = fopen ( ’CToMat ’ , ’ r ’ ) ;

p i p e i n = fopen ( ’MatToC’ , ’w ’ ) ;

count = 0 ;

whi l e ( count < 1005)

rd = f r ead ( pipe out , 5 , ’ char ’ ) ;

r d l = char ( rd ) ;

data recv = s s c a n f ( rd l , ’%d ’ )

i n c da ta = data recv + 1

i n c d a t a s t r = num2str ( i n c da ta ) ;


i n c d a t a s t r d o u b l e = double ( i n c d a t a s t r )

wr i t e da ta = f w r i t e ( p ipe in , i n c d a t a s t r d o u b l e , ’ char ’ )

count = count + 1 ;

end

Appendix B

Code used in the Design - with IAP

// F i l e − Cprocess . c









#d e f i n e PIPE OUT ”CToClient”

#d e f i n e PIPE IN ”ClientToC”

s t r u c t frame {


unsigned char buf [MAX BUF LENGTH] ;

} ;

i n t main ( )

{

76


s t r u c t t imespec s ta r t , end , startcomp , endcomp , startcomm , endcomm ;

double t1 , t2 , d i f f e r e n c e , t3 , t4 , t5 , t6 ;


i n t p ipe out , p i p e i n ;

i n t count = 0 ;

unsigned i n t data send = 0 ;

unsigned i n t r e c v l e n g t h = 0 ;

unsigned i n t r ecv data = 0 ;

i n t i ;



{

f p r i n t f ( s tde r r , ” Cprocess could not open %s\n” ,PIPE OUT ) ;

e x i t (−1);

}


i f ( p i p e i n == 0)

{

f p r i n t f ( s tde r r , ” Cprocess could not open %s\n” , PIPE IN ) ;

e x i t (−1);

}


{

send . buf [ i ] = 0 ;

recv . buf [ i ] = 0 ;

}


c l o c k g e t t i m e (CLOCK REALTIME, &startcomp ) ;


data send = 100 ;



{

s p r i n t f ( send . buf ,”%d” , data send ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &endcomp ) ;


wr i t e ( p ipe out ,&send , (MAX BUF LENGTH)∗ s i z e o f ( char ) ) ;

read ( p ipe in ,& recv ,(4+1)∗ s i z e o f ( char ) ) ;


s s c a n f ( recv . buf ,”%d”,& recv data ) ;

data send = recv data ;

}

c l o c k g e t t i m e (CLOCK REALTIME, &end ) ;

t1 = ( ( double ) startcomp . t v s e c + 1 .0 e−9∗startcomp . tv ns e c ) ;

t2 = ( ( double ) endcomp . t v s e c + 1 .0 e−9∗endcomp . tv ns e c ) ;


t4 = ( ( double ) end . t v s e c + 1 .0 e−9∗end . tv n s e c ) ;



p r i n t f (” Computation time Cprocess = %.7 f \n” , t2−t1 ) ;

p r i n t f (” Cprocess to c l i e n t communication s t a r t time = %.7 f \n” , t5 ) ;


p r i n t f (” C l i en t to Cprocess communication end time = %.7 f \n” , t6 ) ;

p r i n t f (” Cprocess communication time = %.7 f \n” , t6−t5 ) ;

p r i n t f (” Total time = %.7 f \n\n” , t4−t3 ) ;

r e turn 0 ;

}

// F i l e − c l i e n t . c

#inc lude <arpa / i n e t . h>

#inc lude <n e t i n e t / in . h>


#inc lude <sys / types . h>

#inc lude <sys / socke t . h>







#d e f i n e PORT 32000


#d e f i n e PIPE IN ”CToClient”

#d e f i n e PIPE OUT ”ClientToC”

s t r u c t frame {


char buf [MAX BUF LENGTH] ;

} ;

i n t main ( i n t argc , char ∗∗ argv )

{

s t r u c t t imespec s ta r t , end , s t a r t i a p , endiap , startcomm , endcomm ;


double t1 , t2 , d i f f , t3 , t4 ;


s t r u c t sockaddr in c l i addr , servaddr ;

i n t fd , s l e n ;

i n t p ipe in , p ipe out ;

i n t i ;

char ∗ s e r v e r = ” 1 9 2 . 1 6 8 . 1 . 1 ” ;

i n t r e c v l e n g t h = 0 ;

i n t r ecv data = 0 ;

i n t count = 0 ;

i n t n1 = 0 ;

i n t n2 = 0 ;


{

send . buf [ i ] = 0 ;

recv . buf [ i ] = 0 ;

}

s l e n = s i z e o f ( servaddr ) ;

fd = socket (AF INET ,SOCK DGRAM, 0 ) ;

memset ( ( char ∗)& c l i addr , 0 , s i z e o f ( c l i a d d r ) ) ;

c l i a d d r . s i n f a m i l y = AF INET ;

c l i a d d r . s i n addr . s addr = hton l (INADDR ANY) ;

c l i a d d r . s i n p o r t = htons ( 0 ) ;

bind ( fd , ( s t r u c t sockaddr ∗)& c l i addr , s i z e o f ( c l i a d d r ) ) ;


memset ( ( char ∗) &servaddr , 0 , s i z e o f ( servaddr ) ) ;

servaddr . s i n f a m i l y = AF INET ;

servaddr . s i n p o r t = htons (PORT) ;

i n e t a t o n ( se rver , &servaddr . s i n addr ) ;


i f ( p i p e i n == 0 )

{

f p r i n t f ( s tde r r , ” C l i en t could not open %s ” , PIPE IN ) ;

e x i t (−1);

}



{

f p r i n t f ( s tde r r , ” C l i en t could not open %s ” ,PIPE OUT ) ;

e x i t (−1);

}


{

read ( p ipe in ,& recv , ( 5∗ s i z e o f ( char ) ) ) ;


c l o c k g e t t i m e (CLOCK REALTIME, &s t a r t i a p ) ;

sendto ( fd ,& recv , 5 , 0 , ( s t r u c t sockaddr ∗)&servaddr , s l e n ) ;


recvfrom ( fd ,&send , 5 , 0 , ( s t r u c t sockaddr ∗)&servaddr ,& s l e n ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &endiap ) ;


wr i t e ( p ipe out ,&send , ( 5∗ s i z e o f ( char ) ) ) ;

}

c l o s e ( fd ) ;


t3 = ( ( double ) s t a r t i a p . t v s e c + 1 .0 e−9∗ s t a r t i a p . tv n s e c ) ;

t4 = ( ( double ) endiap . t v s e c + 1 .0 e−9∗endiap . tv n s e c ) ;


p r i n t f (” Cprocess to c l i e n t end time = %.7 f \n\n” , t1 ) ;

p r i n t f (” C l i en t to Cprocess s t a r t time = %.7 f \n\n” , t2 ) ;

p r i n t f (”IAP t o t a l time = %.7 f \n\n” , t4−t3 ) ;

r e turn 0 ;

}

// F i l e − s e r v e r . c

#inc lude <arpa / i n e t . h>

#inc lude <n e t i n e t / in . h>


#inc lude <sys / types . h>

#inc lude <sys / socke t . h>








#d e f i n e PORT 32000


#d e f i n e PIPE OUT ”ServerToMat”

#d e f i n e PIPE IN ”MatToServer”

s t r u c t frame {


char buf [MAX BUF LENGTH] ;

} ;

i n t main ( i n t argc , char ∗∗ argv )

{

s t r u c t t imespec s ta r t , end ;

double t1 , t2 ;


s t r u c t sockaddr in servaddr ;

s t r u c t sockaddr in c l i a d d r ;

s o c k l e n t addr len ;

i n t fd ;

i n t i ;

i n t p ipe in , p ipe out ;

i n t s end l ength = 0 ;

i n t send data = 0 ;

i n t count = 0 ;

i n t n1 = 0 ;

i n t n2 = 0 ;

addr len = s i z e o f ( c l i a d d r ) ;


{


send . buf [ i ] = 0 ;

recv . buf [ i ] = 0 ;

}

fd = socket (AF INET ,SOCK DGRAM, 0 ) ;

memset ( ( char ∗)&servaddr , 0 , s i z e o f ( servaddr ) ) ;

servaddr . s i n f a m i l y = AF INET ;

servaddr . s i n addr . s addr = hton l (INADDR ANY) ;

servaddr . s i n p o r t = htons (PORT) ;

bind ( fd , ( s t r u c t sockaddr ∗)&servaddr , s i z e o f ( servaddr ) ) ;



{

f p r i n t f ( s tde r r , ” Server could not open %s ” ,PIPE OUT ) ;

e x i t (−1);

}


i f ( p i p e i n == 0 )

{

f p r i n t f ( s tde r r , ” Server could not open %s ” , PIPE IN ) ;

e x i t (−1);

}



{

recvfrom ( fd ,& recv , 5 , 0 , ( s t r u c t sockaddr∗)& c l i addr ,& addr len ) ;


wr i t e ( p ipe out ,& recv , ( 5∗ s i z e o f ( char ) ) ) ;

read ( p ipe in , send . buf , ( 5∗ s i z e o f ( char ) ) ) ;

c l o c k g e t t i m e (CLOCK REALTIME, &end ) ;

s s c a n f ( send . buf ,”%d”,& send data ) ;

sendto ( fd ,&send , 5 , 0 , ( s t r u c t sockaddr∗)& c l i addr , addr len ) ;

}


t2 = ( ( double ) end . t v s e c + 1 .0 e−9∗end . tv n s e c ) ;

p r i n t f (” Server time %.7 f \n” , t2−t1 ) ;

r e turn 0 ;

}

%F i l e − Mprocess .m

p ipe out = fopen ( ’ ServerToMat ’ , ’ r ’ ) ;

p i p e i n = fopen ( ’ MatToServer ’ , ’w ’ ) ;

count = 0 ;

whi l e ( count < 1005)

rd = f r ead ( pipe out , 5 , ’ char ’ ) ;

r d l = char ( rd ) ;

data recv = s s c a n f ( rd l , ’%d ’ )


inc da ta = data recv + 1

i n c d a t a s t r = num2str ( i n c da ta ) ;

i n c d a t a s t r d o u b l e = double ( i n c d a t a s t r )

wr i t e da ta = f w r i t e ( p ipe in , i n c d a t a s t r d o u b l e , ’ char ’ )

count = count + 1 ;

end

Incremental Design Migration Support in Industrial Control ......Acknowledgements I would like to...

Documents

Transcript of Incremental Design Migration Support in Industrial Control ......Acknowledgements I would like to...