Faculty of EngineeringGadjah Mada University

System Modeling and SimulationPart 2: System Simulation

6 – Visualisation, Verification, and Validation

Andi SudiarsoMechanical & Industrial Engineering

Simulation Software

Some required features for programming discreet event simulation include:

1. generating random number,2. generating random value from a certain probability

distribution,3. advancing simulation time,4. deciding the next event from a list of events and

executing the correct procedure,5. adding and deleting records, such as customer6. collecting and analysing data,7. generating reports, and8. detecting error or wrong condition, debugging, and

displaying a related error message.

Comparing Languages

SPL (Simulation Programming Languages):GPSS, SIMAN/CINEMA, SIMSCRIPT, SLAM,

ARENA, PROMODEL, ECSL, SIMULA, etc.

High Level General Programming Languages (GPL):FORTRAN, BASIC, VISUAL BASIC, PASCAL,

DELPHI, C/C++, VISUAL C++, CBUILDER, etc.

There are always advantages and disadvantages of using simulation languages compared with general programming languages.

SPL versus GPL (1)

Benefits of using SPL compared with GPL include:1. SPL is designed especially for modeling and

simulation, they have required features and toolsto develop a simulation less code is required and reduce developing time.

2. SPL generally allocates dynamic storage during simulation.

3. Models are easier to be modified if written in a simulation language.

4. SPL has a better error detection facility. Many errors related to the simulation have been identified, an appropriate message is then displayed if occurs.

5. SPL has a good reports facility.

SPL versus GPL (2)

Benefits of using GPL compared with SPL include:1. Many modellers or programmers have already

known GPL and may often use them already.2. A very efficient program in GPL needs less

execution time compared with the one using SPL.3. High level GPL has a greater flexibility compared

with SPL.4. The price of GPL is generally cheaper than SPL.5. A choice of three-phase approach or two-phase

approach (event method).Simulator is a computer package (hardware and software) that can simulate a system with a little or even without programming.

Example of flowchart (1)

Prototype customer-process routine for a single-server queueing system

Schedule an arrival event for the next

customer entity

Begin

Is the server idle?

Make the server busy

Wait until selected for service

Remove this customerentity from the queue

Place this customer entity in the queue

N

Y

A

Example of flowchart (2)

Prototype customer-process routine for a single-server queueing system (cont’d)

Schedule a departure event for this

customer entity

Make the server idle

A

Wait until service is completed

Termination of this customer entity

End

SPL Features (1)

1. General featuresa. modeling flexibility; easy to create, easy to modifyb. model execution speedc. maximum model sized. computer compatibility: microcomputer, work

station, mainframe, computer network, internete. ability to combine discreet and continue models

2. AnimationAnimation is used for communicating the simulation to other users. Almost all animations work in a concurrent mode, i.e. the animation runs at the same time as the simulation.

SPL Features (2)

3. Statistical toolsa. common statistical input distributionsb. empirical distributionsc. several random number generatorsd. analysis of simulation resultse. independent replication of simulation run with

different random sets4. Generating reports

SPL is able to generate standard reports in a short time to show the performance of simulated system, such as utilisation level, queueing size, throughput, makespan, costs. User may specify a non-standard report. High quality graphs, histograms are available.

SPL Features (3)

5. Compatibility issuesOutput data should be compatible with other available software, such as spreadsheet, advanced statistical software, word processing, and presentation software, for further analysis and reporting.

Consumer supportSPL needs support from vendor. Vendor needs to

organise a regular conference or seminar, to provide adequate technical supports, to produce a good user’s manual and text books, to give online supports, simulation examples or demos, free software trials, to help software installation, etc.

Visualisation (1)

Visualisation during a simulation gives a description in relation to activity changes and resources utilisation. The benefits of having visualisation include:1. to validate the simulation, helping to check the logic

of simulation and the elements in more detail,2. to know the interaction between the elements and to

identify a possible problem that occurs in the system, for example long queue, low level of utilisation, etc.

3. to help the user to understand and to interpret the simulation, to attract user’s attention.

Visualisation (2)

Developing an interactive facility in a visual simulation gives the user a chance to involve further by changing one or more decision variables and to learn the effects of the changes to the system responses and performances.There are available different methods to visualise simulation. The simplest one uses trace statements. This method gives visualisation of one or more relevant parameter values during the course of simulation. Attention should be given to the possibility of rapid changing of the parameter values so that it causes difficulty in reading and following the values.

Visualisation (3)

A histogram can be displayed during the simulation and any extreme value outside the normal range can be highlighted. However, a histogram does not provide detail logic of simulation.

Recent simulation software generally uses a dynamic iconic display. Every entity has a symbol, a small icon, to represent it in the simulation. This can move around the background during the simulation. The background is made as realistic as possible and describes the simulated system. The use of colour can enhance the quality of animation and may be used for representing different attributes of an entity.

Visualisation (4)

A combination of dynamic iconic display and interactive facility produces Visual Interactive Simulation (VIS). Recent Simulation Programming Language (SPL) has VIS facility. Some languages even use a high resolution graphical interface to facilitate a real visual animation.Recent SPL also provides standard visualisation routines to minimise developing or programming time. The programmer needs to specify the location on the screen and the interconnection between the entities to perform the desired system.

Designing Visual Simulation

To design a visual simulation, all necessary details need to be included to describe the simulation. Pictures or icons used should be as realistic as possible so that the users could easily understand the process or the system.

To design a visual simulation, we need to:1. design the static background,2. design the dynamic elements or entities,3. integrate all the elements as a system.The static background should show all static elements of the simulation and all necessary texts or descriptions. Dynamic elements change according to the related events during the course of simulation.

An example (1)

Visualisation of a warehouse model (from ProModel)

An example (2)

Adding some texts and descriptions

An example (3)

Using interactive facility to change the setting

An example (4)

Running the simulation …

An example (5)

An example of simulation results

Verification and Validation (1)

One of the most important and difficult tasks in building simulation is verification and validation of simulation model.

Verification is concerned with building the model right whereas validation is concerned with building the right model (Banks et al., 2001).

Verification is utilised in the comparison of the conceptual model to the computer representation. • Is the model implemented correctly in the computer?• Are the input parameters and logical structure of the

model correctly represented?

Verification and Validation (2)

Validation is utilised to determine that a model is anaccurate representation of the real system. It is usually achieved through the calibration of the model, i.e. an iterative process of comparing the model to actual system behaviour and using the discrepancies between the two to improve the model. This process is repeated until model accuracy is acceptable.

Validation should not be seen as an isolated set of procedures that follows model development. It is an integral part of model development.

Model building (1)1. Observing the real system and the interaction among

its various components and collecting data on its behaviour.

2. Construction of a conceptual model, i.e. a collection of assumptions on the components and structure of the system and hypotheses on the values of model input parameters.

3. Conceptual validation, i.e. the comparison of the real system to the conceptual model.

4. Translation of the operational model into a computer recognizable form – a computerized model.

5. Verification and validation, rebuild the model if necessary

Model building, verification, and validation

Model building (2)

Verification of models (1)The purpose of model verification is to assure that the conceptual model is reflected accurately in the computerized representation.

1. Have the computerized representation checked by someone other than its developer?

2. Make a flow diagram that includes each logically possible action a system can take when an event occurs and follow the model logic for each action for each event type ACD

3. Closely examine the model output for reasonableness under a variety of settings of the input parameters. Have the computerized representation print out a wide variety of output statistics?

Verification of models (2)4. Have the computerized representation print the input

parameters at the end of simulation, to make sure these parameter values have not been changed inadvertently?

5. Make the computerized representation as self-documenting as possible. Give a precise definition of every variable used and a general description of the purpose of each major section of code.

6. If the computerize representation is animated, verify that what is seen in the animation imitates the actual system.

7. Debugger or interactive run controller (IRC) is an essential component to check mistakes and errors.

Verification of models (3)8. Graphical interface are recommended to accomplish

verification and validation. It is essentially a form of self-documenting and simplifies the task of model understanding.

9. The use of trace, i.e. a detailed computer printout which gives the value of every variable, every time one of these variables changes in value.

Two sets of statistics that can give quick indication of model reasonableness are current contents and total count. Current contents refer to the number of items in each component of the system at a given time. Total count refers to the total number of items that have entered each component of the system by a given time.

Validation of models (1)Validation is the overall process of comparing the model and its behaviour to the real system and its behaviour. Calibration is the iterative process of comparing the model to the real system, making adjustments to the model, comparing the revised model to the real system, making additional adjustment, comparing again, and so on.Comparison of the model to the real system is carried out by a variety of tests. Subjective tests usually involve people and their knowledge about one or more aspects of the system to make judgements about the model and its output (Turing test). Objective tests use appropriate statistical tests, such as t-test.

Validation of models (2)1. Develop models with high face validity, i.e. model

that appears reasonable on its face, by consulting persons knowledgeable about system behaviour on model structure, model input and output.

2. Use any existing information from previous research, observation, and experience.

3. Conduct simple statistical tests of input data for homogeneity, randomness, and goodness-of-fit to assumed distributional forms.

4. Conduct a Turing test by asking knowledgeable people to compare the model output to system output and attempt to detect the difference.

Validation of models (3)5. Compare the model output and the system output by

means of statistical tests.6. After model development, collect new system data

and using the new data, repeat Step 3-5.7. Revise or redesign the model if necessary.

It is often too difficult, costly, and time consuming to use all validation techniques for every model that is developed. Therefore, it is important to choose validation techniques that are most appropriate to assure model accuracy and credibility.

Simulation for decision making

The main purpose of developing a simulation is used for supplying information to make a decision. For example, to decide the number of required machines in a shop floor, the number of cashiers in a bank, etc.

There are models available for decision making as follows.

1. One-off modelsManagerial staff often uses a single-use model to make a decision. For example, to decide which equipment to buy, to rearrange facilities for customers, to decide which product needs to choose from a range of available products, etc.

… for decision making (2)Recent simulation packages support a rapid developing time with a range of available menus, visualisation facilities and ‘easy to use’ interface. On the other hand, computers become faster and more affordable. All of these support a greater role of simulation in the future.

2. Models for a long term useIn a company, there are usually some decisions that need to be made periodically and repeatedly. For example, to decide yearly budget based on recent performance and forecasting, to plan available resources, to manage road traffics that involves one-way roads, traffic lights, road works, etc.

… for decision making (3)The models for this purpose have to be flexible and robust to the changes of inputs, activities and policies. The models are integrated with related data and information to create a decision support system.

3. Cost modelsCosts have an important effect to a decision. Using a simulation, it may be possible to calculate the costs based on the outputs generated during a simulation.Costs are formulated using cost models. The available resources can give an estimation of the fixed costs and the planned activities can produce an estimation of the variable costs.

Decision support system (DSS)

Simulation as part of decision support system

Simulation

Visual outputs

Analysis to find activity

times

Analysis to find starting conditions

Resource levels

and policies

Statistical analysis

Financial analysis

Statistics

Costs

Standard/animated

Data-base

Types of simulation (1)

Several types of simulation are described here.1. Predictive simulation• This simulation is done for prediction or forecasting.• Reliable data (historical and actual) are important to

achieve a good accuracy.• It normally needs a time to achieve a steady state

and several (trial) runs are required to get an accurate estimation.

• The accuracy of results can be improved with variance reduction methods.

• Time series analysis and regression technique are useful to analyse the results.

Types of simulation (2)

2. Comparative simulation• This simulation is useful for deciding whether an

option (choice) is better than another option, based on certain criteria.

• Reliable input data are important to produce reliable simulation outputs for decision making.

• Statistical hypothesis technique can be used for analysing the results.

Types of simulation (3)

3. Investigative simulation• This type of simulation is to investigate the effect of

any change to the system performance.• Visual interactive technique is generally suitable to

be implemented in this type of simulation.• The results can be displayed in a histogram and can

be analysed using statistical tools.

An example (1)

Input – output diagram for general queueing system

Simulation

Inputs:

- available resources

- entity arrival rates

- activity time distributions

- entity characteristics- decisions on

branching (if any)- starting or initial

conditions

Outputs:

- resourcesuse (utilisation)- queue lengths

- queueing times

- states of entities

An example (2)

Input – output diagram of hospital system with underlined decision variables

Simulation

Inputs:

- patient arrival rates

- length of stay distributions

- starting conditions- available

beds- theatre opening

times

Outputs:

- bed utilisation- queue lengths for admission

- queueing timesfor operation

An example (3)

Input – output diagram of hospital system where the objective is ‘given a certain allocation of money, determine the best number of

beds and theatre sessions to treat the most patients’

Simulation

Inputs:

- patient arrival rates

- length of stay distributions

- starting conditions- available

money

Outputs:

- number of beds- theatre opening

times- number of

treated patients

Voluntary presentation (+5 points)Saturday, June 5th 2010, 10 am, M7

Group presentation of the simulation project. Please include the following:

1. aims and objectives,2. assumptions,3. ACD,4. selecting distributions,5. plant layout,6. real system simulation,7. experiment with the system,8. analysis and discussion,9. conclusions, and10. references.

Register your group before presentation (max. 10 groups)!