University of Manchester

School of Computer Science

A Cat Behaviour Simulator

Joshua Langley

Supervised by Ulrike Sattler

April 2016

Abstract

Simulating animal behaviour may lead to advances in our understanding ofhuman and animal interactions with the environment and each other. This couldlead to advances in the fields of Artificial Intelligence, Medicine and Sociologyto name a few. The project explores the possibility of Agent Based Modellingsolutions as applied to animal behaviour to simulate the social and territorialbehaviours present in cat physiology. In particular by using real world mapsand data to better approximate the reality of the system. The approach exploitsemergent behaviour by implementing simple micro behaviours for each cat andan intelligent behaviour transition system using known methods.

The design of such a system is outlined as well as the implementation,focusing on the overall architecture of the system as well as behaviour tran-sition models used to switch between various micro-behaviours. The criterionfor a successful model will be based on statistical analysis of the output datacompared with the real world counterpart. The final output of which will bea piece of software capable of modelling basic cat interactions and the macroenvironment.

Contents

Introduction 51.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Existing Approaches for Modelling Animal Behaviour . . . . . . 5

1.2.1 State Machine Modelling . . . . . . . . . . . . . . . . . . 61.2.2 Hierarchical Task Networks . . . . . . . . . . . . . . . . . 6

1.3 Aims and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . 6

Background Knowledge and Preliminaries 82.1 Cats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Emergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 Agent-Based Modelling . . . . . . . . . . . . . . . . . . . . . . . 9

2.3.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.2 Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3.3 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3.4 Behaviours . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3.5 Environment . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4 Finite State Machine . . . . . . . . . . . . . . . . . . . . . . . . . 112.5 Markov Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.5.1 Hidden Markov Model . . . . . . . . . . . . . . . . . . . . 12

Design 143.1 Goals of this Project . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1.1 Functional . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.1.2 Non-Functional . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2 Development Methodology . . . . . . . . . . . . . . . . . . . . . . 153.2.1 Iterations . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.2 Model View Controller (MVC) . . . . . . . . . . . . . . . 16

3.3 Model Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3.1 Behaviours . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.4 Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4.1 Language . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4.2 Map API . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.5 UI Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.5.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1

Implementation 224.1 Modelling a Cat . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.1.1 State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.1.2 Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.1.3 Behaviour Transition . . . . . . . . . . . . . . . . . . . . . 22

4.2 Implementing the Interface . . . . . . . . . . . . . . . . . . . . . 234.3 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Testing and Evaluation 255.1 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.1.1 Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . 255.1.2 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . 26

5.2 Performance and Scalability Testing . . . . . . . . . . . . . . . . 26

Results 286.1 Overview of System . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 28

Conclusion 307.1 Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

7.2.1 Pathfinding . . . . . . . . . . . . . . . . . . . . . . . . . . 307.2.2 Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . 307.2.3 Other Behaviour Transition Models . . . . . . . . . . . . 31

Appendices 32

A Cat Tracker Case Studies 33A.1 Case Study - Willy . . . . . . . . . . . . . . . . . . . . . . . . . . 33A.2 Case Study - Orange . . . . . . . . . . . . . . . . . . . . . . . . . 34A.3 Case Study - Smokey . . . . . . . . . . . . . . . . . . . . . . . . . 35A.4 Case Study - Oliver . . . . . . . . . . . . . . . . . . . . . . . . . 35

Glossary 37

2

List of Figures

1.1 An example of different paths to complete a goal in a HTN . . . 62.1 General architecture for an ABMS . . . . . . . . . . . . . . . . . 102.2 Example of a FSM for a vending machine. . . . . . . . . . . . . . 122.3 An example Hidden Markov Model. Retrieved from Wikipedia [15] 133.1 Diagram of the MVC architecture pattern . . . . . . . . . . . . . 163.2 Model Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 UI design for the final software. . . . . . . . . . . . . . . . . . . . 205.1 How performance scaled as agents were added to the system . . . 265.2 Image showing the operation of software with 25 cats. . . . . . . 276.1 Image of final program simulating 3 cats. . . . . . . . . . . . . . 28

A.1 Path of Willy the Cat . . . . . . . . . . . . . . . . . . . . . . . . 33A.2 Path of Orange the Cat . . . . . . . . . . . . . . . . . . . . . . . 34A.3 Path of Smokey the Cat . . . . . . . . . . . . . . . . . . . . . . . 35A.4 Path of Oliver the Cat . . . . . . . . . . . . . . . . . . . . . . . . 35

3

List of Tables

2.1 Types of Markov Models, from Wikipedia [16] . . . . . . . . . . . 125.1 Test computer specification. . . . . . . . . . . . . . . . . . . . . . 27

4

Introduction

1.1 Motivation

Animals exhibit incredible variation in behaviour and are capable of creating asequence of events to achieve particular goals. Being able to do this is a raretrait among most biological organisms and would be useful to be able to mimicthis ability to examine current systems and build upon them. Cats in particularhave two properties which are of interest, namely a highly individual behaviourand a high level of influence over the degrees of freedom of other cats via theexpansion and defence of their own individual territory. This level of interactiondespite the cats having a limited knowledge of the environment and the otherterritories may be powerful for modelling other systems.

Creating an application which is able to simulate behaviour is a particu-larly useful area of research with many medical applications as well as economic.Such technology could be applied to a wide variety of fields, example include:simulation of vector-borne disease spread to help combat epidemics and takeprecautionary measures to prevent an outbreak, traffic simulation to create anoptimum transport network for a given location and potentially modelling com-plex human behaviour using a combination of other techniques.

The main challenge in this project is the implementation of systems that arenot fully understood. Having the ability to implement imperfect informationsystems would be useful as it would allow us to implement the behaviour ofa wide range of animals including humans as well as systems which are non-biological in nature. Advances in this field could lead to a greater understandof animals and other systems which may eventually lead to improved ArtificialIntelligence approaches as well.

1.2 Existing Approaches for Modelling AnimalBehaviour

Currently existing approaches rely heavily on stochastic1 models, as these mod-els can be created from mere observation allowing a developer to implement a

1Stochastic methods are those that rely on a probability distribution and a randon variableto determine the outcome and may not have the same output every time.

5

system despite having imperfect information. The techniques below stem fromgame theory and as such use the terminology of an agent to describe the objectbe modelled in this case cats.

1.2.1 State Machine Modelling

Behaviour can be modelled as an ordered list of game states for a particularagent. Th.ese can be arranged with stochastic or conditional transitions be-tween the states based the observed state and current behaviour of the agent.The way an agent moves between these game states is known as a behaviour,however complex agents can exhibit many behaviours to better approximatetheir counterpart systems. While the stochastic approach works for a simpleAgent-Based processes they often struggle to emulate accurate decision mak-ing.This is because it does not rely on the internal state of the agent nor thesituation the agent is currently involved in, thus a conditional state modellingapproach may be more appropriate.

1.2.2 Hierarchical Task Networks

This approach is complex and involves breaking down the game states in to tasksand unravelling them into a network which can be transversed based on thesituational data of the agent and its internal state. Hierarchical Task Networks(HTN) have many advantages when compared to alternatives as it allows theagent to create a plan multiple steps into the future and behaviours can merelybe broken down into specific tasks. For example, eating is made up of manytasks with many possibilities to fulfil the constraints of the behaviour:

GoHome→ GetFood→ EatFood

OR

GoShop→ GetFood→ FindP laceToEatFood→ EatFood

Figure 1.1: An example of different paths to complete a goal in a HTN

While this allows an agent to plan ahead and compensate for mistakesHTNs are typically costly to construct and difficult to unravel into individualtasks[11].

1.3 Aims and Objectives

During the duration of this project I aim to develop an understanding of theprinciple of modelling systems which are imperfect information systems by con-sidering only the observed states and actions using the existing techniques listedabove. I aim to do this by implementing a system which is capable of represent-ing cats and their respective behaviour onto an interactive map and have them

6

interact with each other and the world. The implementation will model a varietyof cat behaviours and model a similar system to their real world counterpartsby socialising and generating territory.

This is particularly challenging as the underlying behaviour rules are notwell understood so must be implemented purely from observation. It all presentsa challenge is discerning the nature in which animals select activities and goalswith many currently existing varied approaches.

7

Background Knowledge andPreliminaries

This section will outline the tools technologies and knowledge used to completethe objectives and requirements of my project. Each will be discuss the motiva-tions for usage and provide background knowledge and terms used throughoutthe remainder of this report.

2.1 Cats

Cats exhibit various behaviours which range greatly in scope, some of whichwill affect other cats such as generating their territory. For the purposes ofthis project I will focus primarily on behaviours which will have a macroscopicimpact on the environment however operate on an individual basis. For exampleeating and sleeping have drastic impacts on the macroscopic behaviour and haveclearly defined objectives, this may also give opportunities to other cats to entertheir territory. It is worth noting that when implementing cat behaviour thatcats do not always make sensible decisions and not always the most beneficial orsuccessful. Cats primarily have a social and predatory instinct which can be seenthrough their interaction with humans and each other, inter-cat interactions canoften be violent if another cat has intruded on the others’ territory. Cats willreturn home as it provides a guaranteed source of food, however they also huntprey outdoors in urban, sub-urban and rural areas. Most of the cat’s needs areprovided for via their owners, in the case of domestic cats, they spend 15 hoursa day sleeping and will opt to sleep well within their territory when possible[10].

2.2 Emergence

Emergence describes the phenomena that is observed when the interactions be-tween smaller entities and components within a system have a macroscopic effectinfluencing the overall system. Emergence or Emergent Behaviour is a usefulproperty particularly when it comes to studying and implementing systems with

8

many individual entities in the real-world. This is due to its simplicity in imple-mentation as developers only have to implement a few basic rules to see widespread effects. Examples of emergence can be seen in nature fairly regularly forinstance in flocks of birds and bacteria cultures. While the system is exhibitingparticular properties it is only as a result of the simplistic rules that the organ-isms follow. The same behaviour is true for human systems such as economiesand markets whereby traders are acting in their own interest but resulting in aglobal change of the market; due to its widespread occurrence it is commonlyused to model biological systems[7, 14].

2.3 Agent-Based Modelling

2.3.1 Definition

An Agent Based Model(ABM)[8] is a subset of algorithmic models typicallyused to model a system with interactions with unique entities(agents) usuallythrough simplistic rules and relying on the emergent behaviour of the overallsystem. Usually used when there is little knowledge of a complete mathematicalfunction and can be adapted to use a variety of game space search algorithms aswell. Instead we create agents with a given state and define simple interactionsand behaviour rules with other agents and their environments. The fundamentalconcept behind such a system is that simple rules are capable of generatingcomplex behaviour and can be extended to approximate real life models usuallyto a sufficient degree of accuracy.

9

Figure 2.1: General architecture for an ABMS

2.3.2 Agents

There exists no formal definition as to what an agent is, however, it is widelyagreed that and agent must have state information describing its situation andwhich will influence its behaviour in the future as well as a set of rules toinfluence its current state [9, p. 2]. Agents must have behaviours described vialow-level routines or via adaptive intelligence using systems such as HTN 1.1which may evolve over time by evolving the network based on the success rateof particular paths through the network.

More complicated implementations of agents include many other compo-nents to better approximate real-world data and create more emergent be-haviour. Memory is often used to represent the agents past experience withother agents and the environment to learn and change their behaviour based onits past experiences. This relies heavily on creating a machine learning modelfor creating better behaviour rules. Implemented biological models tend to relyon genetic algorithms for this task as it accurately documents how animals learnfrom their mistakes however it poses the problem of adjusting the behaviour toaccurately model a specific organism as these algorithms will tend to what ismost successful which is not always the case with animals.

10

2.3.3 Actions

This refers to the low level actions that an agent is able to do at a particularpoint in time such as moving, interacting or idling. These represent how an agentwill interact with the environment and other agents. Actions use resources toperform to reduce the tendency to repeat an action indefinitely and to betterapproximate organisms in the real world.

2.3.4 Behaviours

Behaviours are routines of actions to perform a meaningful task or goal. Be-haviours can be static or unchanging but can also include models which evolvewith the agents experiences. This can be done using various machine learningmethods like the aforementioned genetic algorithms as well as using HTNs togenerate the behaviours by pathfinding through the network.

2.3.5 Environment

The environment is simply everything that is not the agent and can be interactedwith, some ABMS models heavily rely on the environmental model to simulatecertain properties of the agents. Particle systems in particular interact withtheir environment regularly to influence their motion and emulate pressure onthe surroundings. The environment may also contain other information such asparticular rates for certain areas pertaining to the success of the agents withinit, and can directly impact the decisions made by the agents.

2.4 Finite State Machine

A Finite State Machine is mathematical model which describes states and tran-sitions between said states. Many variations of the basic framework exist, someof which will be discussed later in the report, each with their own advantagesand disadvantages. This model provides the benefit of being simplistic in natureand allowing for transitions based on observation from the real world by usingthe frequency of observed states to generate a probability table.

11

Figure 2.2: Example of a FSM for a vending machine.

It is also possible to implement a FSM such that transitions are triggeredby conditions instead of a stochastic process which may be more appropriatefor particular systems especially those which are understood in their implemen-tation. This is more difficult to implement as the system grows in size howeveras it relies on being able to accurately identify conditions for state transitionswhich may be difficult for the transitions between 100 different states whichcould have as many as n2 transitions which for a 100 state system would requirethe implementation of 10,000 conditions.

2.5 Markov Models

Markov models are a stochastic subset of Finite State Machine used to imple-ment systems for which the Markov Property holds. Stating that the next stateonly relies on the current state and no previous states. Markov models are gen-erally classified into 4 different classes based on the system being implemented.

Perfect Information Non-Perfect Information

Autonomous Markov Chains Hidden Markov ModelControlled Markov Decision Process Partially-observable Markov De-

cision Process

Table 2.1: Types of Markov Models, from Wikipedia [16]

2.5.1 Hidden Markov Model

”An HMM is a doubly stochastic process with an underlying stochastic processthat is not observable (it is hidden), but can only be observed through anotherset of stochastic processes that produce the sequence of observed symbols” [13].They can be described as a specific Markov Model used to model autonomoussystems where the state of the system is only partially visible or not visible at all.This is performed by utilising the inference of state from the observation of realworld data to create a hidden finite state machine that is capable of generating

12

accurate observations. The general architecture of a HMM consists of statesand observations with each state having an associated emission probability toeach of the possible observed states such as in the diagram below:

Figure 2.3: An example Hidden Markov Model. Retrieved from Wikipedia [15]

This has been used extensively for modelling biological processes [5] due toits ability to be trained from observation. This allows them to be trained easilywith an incomplete knowledge of the system, some studies have used extensionsof this architecture to improve on this model.

2.5.1.1 Kalman Filters

Kalman Filters are a form of Hidden Markov Models where that the observedstate is instead represented as a normal distribution and introduces error to theobservations based on the selected probability distributions. Sophisticated mod-els have used this technique to model the traffic of crowds of humans throughthe environment [12]. Showing relative success when compared with other mod-els and being able to represent the error during the observation process whichprovided better inference to the underlying state.

13

Design

3.1 Goals of this Project

To better plan ahead for the project I specified various requirements which mustbe upheld listed below:

3.1.1 Functional

FR-1 My simulator must be able to simulate simple single agent cat behaviouron the scale of a map.

FR-2 My implemented model must be able to represent some sort of relationshipwithin a multi-agent environment.

FR-3 The User must be able to place a property onto the map and it must berepresented in some visual manner.

FR-4 The model must support cats being able to have a home property with anassociated territory around it.

FR-5 Cats need to perform the basic actions of a cat e.g. Move, Eat, Sleep etc.

FR-6 My project must be able to simulate a wide variety of cats such as catswho are more aggressive or timid.

3.1.2 Non-Functional

NFR-1 It should be able to represent a large number of cats successfully in real-time. (25 cats)

NFR-2 It should be able able to approximate cats to a small error margin. (0.5σ)

NFR-3 It should be able to handle with dynamic open world food sources.

NFR-4 It should be able to model distractions i.e. suddenly switching behaviouron transit home.

14

NFR-5 It should be able to use geo-data from accessible sources to help improvethe simulation and variety of the behaviour.

NFR-6 Two or more cats should be able to be assigned to the same property.

3.2 Development Methodology

Incremental development will be used throughout the project to quickly obtaina working implementation of the software, as such have divided the developmentinto 3 smaller iterations delivering incremental improvements. To ensure a highlevel of code quality I have included time to perform a code review to ensure thatthe implemented model is sound implemented correctly. This code review will beperformed between iterations 2 and 3 mainly focusing on the finite state machineimplementation as this will be expended later. Documentation will be providedto the 3rd party performing the code review to improve their understanding ofthe model.

3.2.1 Iterations

3.2.1.1 Iteration 1: Basic Interface and Structure

Iteration 1 focuses on the general project architecture and on implementing thebasic interface to interact with the model for testing purposes. This requires theimplementation of the map API to be completed and the basic MVC architectureto interact with the underlying model. However, this does not require theimplementation of a fully functional model however will include a basic agentdescription. After completing this I will be able to focus solely on the modeland various implementations for behaviour transition models.

3.2.1.2 Iteration 2: Random behaviour selection

Initially I will focus on implementing a random behaviour model which will selectthe behaviours on a simple stochastic process. This requires the implementationof the behaviours I plan to use in their basic form 1 to be expanded later to bemore complex.

3.2.1.3 Iteration 3: Finite State Machine

This iteration will focus on getting a initial working version of the underlyingmodel to ensure that progress can be made to enhance the model while stillbeing able to evaluate the performance of this weaker model. This will alsoform the starting point for the next iteration which will build upon the finitestate machine model

1see 3.3.1

15

3.2.1.4 Iteration 4: Hidden Markov Model

This will involve extending the previous iteration to allow for hidden statesand an additional probability table to create the emission probabilities of anobservation. Implementation should remain relatively trivial with most of thetime being spent correcting the probabilities from the real world data and findingappropriate source for this data.

3.2.2 Model View Controller (MVC)

This is a design pattern that features heavily throughout my design, the basicprinciple of which is to separate the user’s interactions from the model andintroduce an intermediate entity to handle user input and thus update the modelappropriately. The view then simply interpret the contents of the underlyingmodel to feedback to the user. This has the advantage of separating user inputfrom the underlying model thus protecting it from erroneous input and providinga common interface to interact with the model from many views if needed.

Figure 3.1: Diagram of the MVC architecture pattern

In the case of this project the underlying model is made out of an agenthandler class which will update the agents and provide an interface betweenthe environment and the individual agents. This model will then be displayedonto the main UI and can be viewed in an appropriate way by the user who caninteract with the system to see the effect on the underlying model.

16

3.3 Model Architecture

Figure 3.2: Model Architecture

My model will follow this architecture with the Simulator class being responsi-ble for updating the agents and acting as interface between the agent and theenvironment to ensure that a particular action is legalvalid. Each cat will havean attached behaviour model which will be responsible for deciding the currentbehaviour of the animal i.e. whether the cat should be sleeping, eating explor-ing etc. based on the state of the cat. The behaviour model will then call thecorrect behaviour to perform the actions on the cat and its environment as wellas other cats.

3.3.1 Behaviours

Cats typically exhibit potentially hundreds of both microscopic and macroscopicbehaviour which would be far to ambitious to implement in the time frame ofthe project. As such I decided to instead focus on the macroscopic behavioursonly and have narrowed it down to the following behaviours and routines:

Eating This behaviour outlines the routine to eat and is to be the more likelybehaviour as the cat grows hungry over time. This behaviour may beextended to include later to eating from dynamic food sources which couldbe hunted. However the initial implementation will look only at eatingfrom its own property.

Sleeping Sleeping can be performed anywhere however, cats will tend to sleep onlyin areas where they feel safe ie. from within their territory. So the catagents will tend to sleep in and around their assigned home.

Exploring Exploring will include travelling to neighbouring territories and to areaswhich the cat has not visited before or recently.

17

Territorialise This is to represent the cat both gaining new territory and protectedits own. However the final implementation will have no global map ofterritory instead each agent will only have a view of its own territory. Thefinal global territory will be the emergent behaviour generated from thecats and their respective territories contesting and shifting.

Socialising This is to represent the cat socialising with its owner primarily performedin the property. This can be extended to include cats socialising with eachother in neutral territory.

Fighting This behaviour should be triggered when a cat has entered the currentcats territory to prevent the cat from claiming the territory. This wouldalso trigger a flight or fight response in the recipient cat.

Grooming Grooming can be performed at any time and would appear as thoughthe cat would stop to clean. As cats perform activities the amount ofaccumulated dirt will grow to the point they need to clean themselves orrisk disease.

These were devised from case studies from the CatTracker project [3] to iden-tify any movements and behaviours which might be present as well as fromobservation.

3.4 Technology

3.4.1 Language

3.4.1.1 Requirements

Due to the nature of the system I have decided to implement my language hasto be appropriate for the paradigms discussed previously. The language I choosemust:

• Have Strong Object-Oriented programming abilities, supporting the typ-ical OOP paradigms to help design the system. This is crucial as theunderlying model is inherently object oriented with many instances of oneclass to represent the state of a particular agent.

• Have available map frameworks allowing me to implement a static or dy-namic map UI component that allows me to draw the state of the objecton top. The frameworks must be able to be interacted with by the userto influence the underlying model.

• Be capable of running the underlying model at a sensible speed, preferableover 15 iterations of the system per second.

18

3.4.1.2 Conclusion

For my choice of language I looked specifically at languages supporting object-oriented programming paradigm due to the object nature of agent-based sys-tems. Because of this I looked first at using Java which is well known forenforcing object-oriented programming and also supported a wide variety ofAPIs particularly for geo-coding and maps. As I have used Java in the past andam competent with the language this eventually led to me deciding on usingit for the implementation of my project. It came with many advantages overother languages such as Python, which I also considered, due to its OOP abilityand its widely available libraries for a large variety of tasks and state transitionmodels. However I chose not to use Python due to the language being inherentlyslower than other alternatives.

3.4.2 Map API

3.4.2.1 Requirements

My project requires a map to represent the environment for the cat to existwithin and interact with. To better select a particular map API, I set a numberof well defied requirements to narrow down the large selection. These require-ments are as follows:

• The selected Map API must provide functionality to draw onto the mapand must not limit the number of shapes and objects that could be poten-tially drawn. This is to prevent the development of an overlaid transparentrendering engine to render the agents and properties onto the map.

• The map should be able to render a large variety of locations and scenarios.This is primarily to better evaluate the finished program as it will allowme to test it in a wide variety of situations and environments.

• The API must be able to support being updated at regular intervals toreflect the model and a appropriate rate.

• It must provide a Java library to interface with the map source.

3.4.2.2 Conclusion

Firstly I considered the Google Map Engine which has APIs available for mostlanguages. While this option did give a strong static map mode it did notgive much ability to interact with the map to change locations. It also lim-ited the number of queries that can be made from a particular Google accountwhich may have restricted me during the implementation stage. However it didprovide utilities to rendering shapes onto the map and getting extra potentialuseful features such as location meta-data and building information. However,I later discovered that the Google Maps Engine API was due to be discontinuedJanuary 2016, and subsequently discounted it from my considerations.

19

This led me to look at an Open source map engine known as the Open-StreetMap API [2] which did not place a limitation on the number of requeststhat can be made and provided an available Java library. The Java library wasa Java Swing component which allowed for shapes and polygons to be drawndirectly onto the underlying map layer. It also provided building information.The API in question was called JMapViewer [1] and it provided a Javadoc whichmade it particularly appealing and was able to be interacted with and moved bythe user. Taking this all into consideration, I chose to use the OpenStreetMapAPI.

3.5 UI Design

3.5.1 Design

Java has two main libraries providing UI components for application develop-ment: the Swing API and the new JavaFX API. For this project I plan onusing the Swing API due to its more extensive documentation and tutorials asopposed to JavaFX which has only been included in the Java standard librarysince Java 8. This should allow me to implement a simple functional UI and letme focus primarily on the underlying model driving the cat behaviour as this isa focus of my project.

Figure 3.3: UI design for the final software.

The main focus of this design is to increase the area of the screen takenup by the map. This is mainly because this area will be used to both viewand interact with the underlying model. The list will allow you to select anagent to place onto the map and place them directly onto the map via double

20

clicking on a location. Assigning a property to a cat will be done by placing theagent directly onto the property it is to be assigned to. While not immediatelyobvious this should make interaction very quick as it reduces the number ofbutton clicks and interactions to place agents and properties; it also makes iteasier to place down multiple agents and properties.

21

Implementation

4.1 Modelling a Cat

The class hierarchy I implemented was designed to increase the flexibility of thesystem by utilising abstract classes to allow the map to draw any agent andthe behaviour simulator to run for any implemented agent. Every behaviourimplements and update method which performs a different task on each frameand will update the needs of the cat dynamically.

4.1.1 State

The cats have two parameters that describe its state, the first is its needs whichrepresent the short term state of the cat to have a large impact on the chosenbehaviour. The second set of parameters describe the personality of the catand manifest themselves as a bias to particular behaviours and a reluctance toothers. This allows the model to represent the many different types of cats i.e.lazy cats, active cats, clean cats etc.

4.1.2 Behaviour

Behaviours extend an abstract behaviour class which provides helper methods toaccess the state of the owning cat. These are then passed to a behaviour modelthat extends the behaviour model class and implements a method which getsthe next state, this is so behaviour models can easily be swapped out later. Thesimulator then updates the state of every agent in the environment which in turnmakes the cats get the next behaviour in the implemented behaviour model. Wewould expect most of the time for repeated behaviour between frames and onlyrarely should they switch as the agents are being updated 15 times a second bythe simulator class. Once an agent is in a particular behaviour that behaviourupdates the need parameters of the agent which will later influence the nextbehaviour and gradually reduce the need until the cat switches behaviour.

4.1.3 Behaviour Transition

The first sprint involved implemented a finite state machine model to transitionbetween states with a given probability. This is implemented as a transition

22

table of the states where the transition probability is equal to the followingequation:

P (X,Y ) = (need(Y ) + 1)× 7 + (bias(Y ) + 1.0)× 5)

As the probabilities update the cat is more likely to switch to a particularbehaviour from the current one to the one which is needed the most. This isdone by random number generation which will find a number between 0 and 1and then match that to the probability distribution to find which behaviour toswitch to.

This later evolved to be a Hidden Markov Model trained whose emissionand transition probabilities are hard coded in. The Hidden Markov Modelworks by inferring the internal state from observation as discussed earlier. Thisis implemented via 2 2D arrays for both the emission probabilities and thetransition probabilities where calculated from the following:

P (B | need(1), . . . , need(B))

This made it fairly trivial to implement over a Hidden Markov Model. Theemission probabilities were simply the rates at which cats have been observedto be in a particular state.

4.2 Implementing the Interface

To implement the intended design, I had opted to use the Java Swing frame-work which had many tutorials and excellent documentation. The main UIcomponent was the JMapViewer [1] which was the particular implementation ofthe OpenStreetMap API that I had opted to use due to its Swing integration.As this was the view of the underlying model, the map was to fill the entirecontaining JFrame so to increase the area used. I used a JLayeredPane, thisallowed me to render my own components on top of the map while still allowingthe user to click on the map and drag the component to move the map around.

In the original design the height of the UI list was later changed to be a thirdof the original height, this allowed the user to see more of the map and reducethe amount of wasted space. This was made of a JPanel with a GridLayoutthat had 3 rows and 1 column. The top row contained a Linear Layout whichcontained 2 editable JTextFields and a JButton to allow the user to see thecurrent latitude and longitude that they were centered on and to allow them tomove the map to a radically different location by entering in new coordinates.The second row contained a JScollPane containing a JList which contained thenames of creatable agents to place onto the map. The third row contained aJSlider which allowed the user to change the simulation speed going from realtime to 2 minutes per second to better see the cat interactions.

23

4.3 Challenges

During the project I quickly realised that there was little technical statisticalinformation available regarding cat behaviour. I managed to discover a studywhich did give GPS information about a particular set of cats but was not ableto find any studies which had annotated behaviour information as well. Thismade it particularly difficult to verify the behaviour transition model, to getround this I used a statistical test instead based on the location data which waseasily accessible. This proved that the rough observations were correct evenif the underlying model was not and could be used to replicate the statisticalevidence that has been observed.

The chosen map API [1] did not have many tutorials or examples of its useand as such had to be implemented using the included Java Doc. this made itdifficult to unravel its operation in integrate new functionality. This was neededto allow the map to draw agents onto the map and their respective properties.To accommodate for this, I merely extended the JMapViewer to create my ownMapPanel class to provide the additional functionality. User interaction wasachieved using a JMapController which intercepted user interaction events andallowed me to adjust the functionality of the map. I implemented the placementof agents by detecting double clicks and getting the selected agent from theoverlaid list to make it easier to place multiple points.

24

Testing and Evaluation

5.1 Tests

Testing presented a challenge due to the lack of statistical studies on cats, how-ever I found an open source crowd funded study called CatTracker [3]. Thisprovided open source individual GPS information about the daily lives of catswhich I was able to extract using a simple Javascript function. This could thenbe used to compare with the data generated from my model to perform somestatistical analysis of there movements. I created unit tests which were usedduring the development cycle to ensure new functionality worked as expectedand during integration to ensure existing functionality was not broken. I alsoimplemented performance and scalability tests to ensure I met the project re-quirements of over 25 cats operating simultaneously.

5.1.1 Unit Testing

Unit testing was used to test small methods that were mission critical and thosewhich were often called. These were built continuously throughout the projectto ensure each iterations’ crucial code was fully functional with as few bugs aspossible. Below is a list of unit tests that were used in the project, outliningthe purpose of the unit test:

• Cat Unit test focusing on the cat agent creation and root agent methods.These tests were used to ensure the agents were all are able to interactwith the underlying model. This included 4 tests testing the constructor,move and state interaction methods which were not only crucial for thedescription of the agent for the model but also the correct interface forthe behaviour transition models that were implemented. If these unit testswere to fail it would indicate that problems observed with my programmay instead be with the agent implementation.

• Map interface unit testing to ensure that the cats can interact with themap correctly which presented a difficulties. This is mainly due to theLatitude and Longitude coordinate system that the map uses which causeddouble precision errors.

25

5.1.2 Statistical Analysis

To ensure methodology correctness I had to implement a way of objectivelyidentifying a correct simulation with the limited data available. I opted toperform simple statistical analysis on the data using metrics such as the meanposition and variance between the positions. This produced data which I wasable to compare with the data from CatTracker [3] to see if it generated similarpatterns in the data via analytical statistics, these include the variance, mode,median and range. This will make it easier to compare the two sets of figures.

5.2 Performance and Scalability Testing

The requirement for this test was for the program to be able to render andsimulate the cats behaviour in real time at 15 frames per second. 25 cats wasthe threshold for simulation as this would put adequate strain on the system.I then increased the number of cats until the tests failed which indicated thatI had reached the capacity of any performance bottlenecks. The point of thistest is to ensure that the system can contain enough agents to witness emergentproperties.

Figure 5.1: How performance scaled as agents were added to the system

26

Figure 5.2: Image showing the operation of software with 25 cats.

Above shows the system operating with 25 cats I gradually increased thisby multiples of 5 till the real time constraint was no longer met. This happenedat 55-57 cats on the test machine I used with the specification listed below:

CPU Intel(R) Core(TM) i5-2400CPU Clock Speed 3.1GHz

CPU Number of cores 4 Physical, 8 LogicalCPU Cache Size 6144 MB

Ram Size 7.966 GBRam Speed 1333 MHz

Table 5.1: Test computer specification.

While the machine was not operating at 15 frames per second, a further13-15 cats reduced the frame per second to under 1. This is more than adequatefor the purposes of the final software.

27

Results

Summary

This section outlines the outcome of the project and provide a thorough walkthrough of the final result. Focusing mainly on the end-user experience, imple-mented functionalities, detailing achieved requirements.

6.1 Overview of System

Figure 6.1: Image of final program simulating 3 cats.

6.1.1 Requirements

6.1.1.1 Functional

The final project met all of the functional requirements. A user is able to visiblysee the current behaviour of the individuals in the system. The UI allows theuser to place properties and cat agents onto the map (FR-1, FR-3, FR-4). Tecats themselves have 7 distinct behaviours and have simple relationships withother cats providing interaction.

28

6.1.1.2 Non-functional

The performance tests show that the software is capable of displaying morethan 25 unique agents onto the map. However it is unclear as to the accuracyof the simulation due to the lack of real-world data. However according tothe statistical measurement performed previously it is apparent that the agentsdo reflect the cats when we compare the cats movement only. No food sourceswere implemented other than eating in the cats own property however cats couldbecome distracted during a particular behaviour. Finally, while two cats canbe assigned to the same home they compete for territory as though they liveindependently which suggests an improved relationship model is needed.

29

Conclusion

7.1 Reflection

My progress throughout the project was considerably consistent, and the rateof progress gradually increased over the course of the project which is to beexpected. Had more time been available I would have focused more on the test-ing and various model implementations, moving away from finite state machineimplementations to investigate other approaches.

7.2 Future Work

7.2.1 Pathfinding

Currently cats can walk through houses and other structures, while this doesmatch the GPS data it does not replicate the reality that cats walk aroundthe house, as the GPS only updates every 10 seconds or so. Many pathfindingimplementations exist but most rely on a grid and the agent having a full under-standing of the environment which may not be the case. Instead I would proposeto use an algorithm such as Theta* [6] which is typically used for pathfindingfor robots and is based on line of sight. This is much more appropriate for acontinuous space such as the ones used by Map API (Latitude/Longitude). Thiswould better replicate the fact that the cat may not know the full state of theworld and better replicate a real life scenario.

7.2.2 Kalman Filters

A Kalman Filter is a specific type of Hidden Markov Model where the latentvariables instead are represented as a Gaussian distribution to better simulatean error in the observation. A study was conducted at MIT [12] modelled humantraffic using Kalman filters which proved slightly better than Hidden MarkovModels as it better simulated the statistical noise present in the real world data.

30

7.2.3 Other Behaviour Transition Models

Many models exist which may provide better parallel to real world decision mak-ing in particular I have investigated Decision and Behaviour trees which are ahierarchical implementation of behaviour transitions. These are used primarilyin games to replicate human behaviour to some degree by creating a tree of con-ditions with the leaf nodes representing a particular behaviour. This approachdoes not rely on the emergent behaviour property that ABMS tries to exploit.As such this may create vastly different results which may have a greater level ofsuccess. This is particularly interesting as there exists frameworks to implementbehaviour trees in Java such as the Behaviour Tree Framework [4].

31

Appendices

32

Appendix A

Cat Tracker Case Studies

Below are some interesting example of real world cats from the CatTracker[3] project, I have chosen some examples which exhibit peculiar properties ei-ther due to environment or potentially other cats and animals influencing theirmovement. In each case study we do see repeated behaviour, all cats exhibit twomodes of movement with them moving close and often around a inner undis-puted territory and a further afield outer mode which indicates them movingoutside of their own territory. We can also see repeated movements such ascycles and loops and u-turn manoeuvres. While we can not say for sure thecause of these manoeuvres I speculate that the loops and cycles are exploratorybehaviours and U-turns indicating a cat encounter as the case studies whichshow this behaviour tend to be in populated areas.

A.1 Case Study - Willy

Figure A.1: Path of Willy the Cat

Willy lives in a neighbourhood which is located close to a large road which wecan see from the car park in frame, is well used. We can see that Willy spends

33

the majority of his time moving around one specific property and its immediateneighbours, it is therefore likely to assume that is is its undisputed territory. Inthe few instances he left this immediate area we can see that he move abouttwice the distance away primarily to the north of the property, this is likely dueto the traffic and noise from the road to the south. When he moves outside histerritory we can observe that at some point he turns round completely, likelyindicating an encounter with another cat’s territory. Other journeys shows apath which eventually loops back to his original property which could be anattempt to expand his territory or exploring the surround area.

A.2 Case Study - Orange

Figure A.2: Path of Orange the Cat

This case is particularly interesting as the cat is enclosed by a river which limitshis access to the north, west and south. In comparison with Willy A.1 wecan see a vastly different range of movement with Orange only moving to the2nd closest neighbours of his property and with much of his time being spentby the river. This shows the variation between two different cats with somepreferring to remain isolated and with a much smaller territory when comparedwith others.

34

A.3 Case Study - Smokey

Figure A.3: Path of Smokey the Cat

This shows how the environment affects the cats decisions with Smokey decidingto venture much further to the north due to the large park area which is unlikelyto be within any other cats territory due to the lack of housing, it may also serveas a food source for Smokey. We can also observe that smokey does not venturetowards the south which is more populated by residential housing, this could bea result of other cats territories or a preference for non-populated areas.

A.4 Case Study - Oliver

Figure A.4: Path of Oliver the Cat

Oliver travels much further than the previous case studies which could be theresult of a lack of other cats in the area or it could be he is particularly aggressiveand suppresses the other cats territories. We do not see the U-turn behaviourthat we see in the other paths. However we do see areas which lie within thepath which are left unvisited which could indicate another cat household withinthe exploratory loops. We can say that Oliver is particularly adventurous and

35

moves much further away than the other case studies. The environment couldbe a large contributing factor as this environment has no main roads and isfar greener, with the extra open space it might reduce the chance of a violentencounter allowing Oliver to travel much further.

36

Glossary

ABMS Agent based Modelling Solutions are ones which approximate a com-plex mathematical model by implementing agents with a state and apply-ing simple rules and behaviours to them. 36

Agent A construct which describes a entity that can interact with the worldwith a finite set of interactions and behaviours.. 6, 36, 37

Attribute A parameter that describes the internal state of an Agent within aABM system and one which may influence the behaviour of the agent.. 36

Behaviour The way in which an animal or person behaves in response to aparticular situation or stimulus. In the context of ABMS it also refers toa set of actions that an agent can make at a given time and these routinesmay change over time.. 6, 36

Emergent Behaviour Behaviour emerging from the microscopic componentsof an overall system having a macroscopic effect on the global state of thesystem arising through the interactions between these components.. 8, 36

Finite State Machine A collection of states and transitions which describethe switching between states for a system. The transitions can be imple-mented on a stochastic basis or on a conditional basis.. 12, 36

Global Behaviour The macroscopic behaviour of the entire system emergingfrom the aggregation of local behaviour from its constituent nodes. 36

Hidden Markov Model A Markov model where the system being implementedis a Markov Property with unobserved hidden states. 36

Kalman Filter A type of Hidden Markov Model with a continuous latent vari-ables and where the latent variables and observations follow a Gaussiandistribution.. 30, 36

Local Behaviour A behaviour limited to its owner and its neighbours usuallymentioned in topics involving complex networks. 36

37

Markov Property A property of systems which states that the next stateonly relies on the current state and not any past states. 12, 36, 37

38

Bibliography

[1] Jmapviewer. http://wiki.openstreetmap.org/wiki/JMapViewer, 2015.[Online; accessed November 26th, 2015]. 3.4.2.2, 4.2, 4.3

[2] Openstreetmap. http://wiki.openstreetmap.org/wiki/About_

OpenStreetMap, 2015. [Online; accessed November 26th, 2015]. 3.4.2.2

[3] Cat tracker. http://cats.yourwildlife.org/project-description/,2016. [Online; accessed March 17, 2016]. 3.3.1, 5.1, 5.1.2, A

[4] Java behaviour trees (jbt). https://github.com/gaia-ucm/jbt, 2016.[Online; accessed April 12th, 2016]. 7.2.3

[5] Sean R Eddy. Hidden markov models. Current opinion in structural biology,6(3):361–365, 1996. 2.5.1

[6] Dave Ferguson and Anthony Stentz. Field d*: An interpolation-based pathplanner and replanner. In Robotics research, pages 239–253. Springer, 2007.7.2.1

[7] David G Green. Emergent behaviour in biological systems. Complex sys-tems: from biology to computation, pages 24–35, 1993. 2.2

[8] Catholijn M. Jonker and Jan Treur. Agent-based simulation of animalbehaviour. Applied Intelligence, Volume 15(Issue 2):p. 83–115, September2001. 2.3.1

[9] M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, editors.Tutorial on Agent-Based Modelling and Simulation. 2.3.2

[10] Paul Leyhausen, Barbara A Tonkin, et al. Cat behaviour. The predatoryand social behaviour of domestic and wild cats. Garland STPM Press.,1979. 2.1

[11] Negin Nejati, Pat Langley, and Tolga Konik. Learning hierarchical tasknetworks by observation. In Proceedings of the 23rd international confer-ence on Machine learning, pages 665–672. ACM, 2006. 1.2.2

39

[12] Alex Pentland and Andrew Liu. Modeling and prediction of human behav-ior. Neural Computation, Volume 11(Issue 1):p. 229–242, September 1999.2.5.1.1, 7.2.2

[13] Lawrence R Rabiner and Biing-Hwang Juang. An introduction to hiddenmarkov models. ASSP Magazine, IEEE, 3(1):4–16, 1986. 2.5.1

[14] Peter Wavish. Exploiting emergent behavior in multi-agent systems. ACMSIGOIS Bulletin, 13(3):16, 1992. 2.2

[15] Wikipedia, the free encyclopedia. Hidden markov model, 2016. [Online;accessed April 23, 2016]. (document), 2.3

[16] Wikipedia, the free encyclopedia. Markov model, 2016. [Online; accessedApril 23, 2016]. (document), 2.1

40