Evolution of Evolution of Complex Complex SystemsSystems

Lecture 3: Theoretical Lecture 3: Theoretical foundationsfoundations

Peter Andras / Bruce CharltonPeter Andras / Bruce Charlton

peter.andras@ncl.ac.ukpeter.andras@ncl.ac.uk

bruce.charlton@ncl.ac.ukbruce.charlton@ncl.ac.uk

2

ObjectivesObjectives Meaning Meaning LanguageLanguage MemoryMemory Structure and subsystemsStructure and subsystems Information subsystemInformation subsystem Double contingencyDouble contingency System identitySystem identity Identity violation and adaptationIdentity violation and adaptation ComplexityComplexity

3

Communication systemsCommunication systems

Communication system

Communication units

4

CommunicationCommunication

Sender unit:

Signals generated

Receiver unit:

Signals received

Communication:

Signals transmitted

5

What is the meaning ?What is the meaning ?

How does the sender and receiver How does the sender and receiver attach meaning to the attach meaning to the communication ?communication ?

E.g., mother cat and kitten E.g., mother cat and kitten meowing, courtship dance of birdsmeowing, courtship dance of birds

6

Communication by the Communication by the sendersender

Communication: sequence or pattern of Communication: sequence or pattern of signalssignals

Each signal may be followed by others Each signal may be followed by others with some probabilitywith some probability

The sender selects the continuation The sender selects the continuation signal eliminating all other possible signal eliminating all other possible continuationscontinuations

E.g., sequence of words in human E.g., sequence of words in human speechspeech

7

Communication to the Communication to the receiverreceiver

Communication: sequence or Communication: sequence or pattern of signalspattern of signals

Each signal is followed by others Each signal is followed by others with some probabilitywith some probability

Each received signal eliminates all Each received signal eliminates all other possible continuationsother possible continuations

E.g., hearing human speechE.g., hearing human speech

8

Probabilistic interpretation of Probabilistic interpretation of the meaning – The sender – 1 the meaning – The sender – 1

The sequence of signals is a sample of the The sequence of signals is a sample of the conditional sequence continuation conditional sequence continuation distributions over the signal space (a priori distributions over the signal space (a priori distributions)distributions)

The sample fits the best a possibly different The sample fits the best a possibly different set of conditional distributions over the space set of conditional distributions over the space of possible signals (a posteriori distributions)of possible signals (a posteriori distributions)

The difference between the corresponding a The difference between the corresponding a priori and a posteriori distributions is the priori and a posteriori distributions is the meaning of the communication for the meaning of the communication for the sendersender

9

Probabilistic interpretation of Probabilistic interpretation of the meaning – The sender – 2 the meaning – The sender – 2

0.2 0.1 0.5 0.2 0.6 0.2 0.15 0.05

0.25 0.3 0.15 0.3 0.05 0.6 0.1 0.25

0.2 0.45 0.05 0.3 0.1 0.65 0.15 0.1

0.4 0.1 0.2 0.3 0.25 0.2 0.1 0.45

0.1 0.5 0.2 0.2 0.4 0.2 0.25 0.15

Current A priori A posteriori

10

Probabilistic interpretation of Probabilistic interpretation of the meaning – The receiverthe meaning – The receiver

The sequence of signals is a sample of The sequence of signals is a sample of the expected conditional sequence the expected conditional sequence continuation distributions over the signal continuation distributions over the signal space (a priori distributions)space (a priori distributions)

The sample fits the best a possibly The sample fits the best a possibly different set of conditional distributions different set of conditional distributions over the space of possible signals (a over the space of possible signals (a posteriori distributions)posteriori distributions)

The difference between the The difference between the corresponding expected a priori and a corresponding expected a priori and a posteriori distributions is the meaning of posteriori distributions is the meaning of the communication for the receiverthe communication for the receiver

11

Meaning and informationMeaning and information Meaning: difference between probability Meaning: difference between probability

distributionsdistributions Information: (quantitative) measure of the Information: (quantitative) measure of the

difference between distributions, it may difference between distributions, it may ignore qualitative features and details of ignore qualitative features and details of the differencethe difference

If the communication is made up of very If the communication is made up of very likely continuation signals the information likely continuation signals the information conveyed is littleconveyed is little

The information content is high if there is The information content is high if there is a significant difference from the expecteda significant difference from the expected

12

What is meaningless ?What is meaningless ? If the sequence of signals contains consecutive signals If the sequence of signals contains consecutive signals

that have zero continuation probability the that have zero continuation probability the communication is meaninglesscommunication is meaningless

This may happen also because of communication noise This may happen also because of communication noise (e.g., noisy mobile phone)(e.g., noisy mobile phone)

If a sequence of communications is dominated by If a sequence of communications is dominated by meaningless combinations of patterns / sequences of meaningless combinations of patterns / sequences of signals the communication is meaninglesssignals the communication is meaningless

Meaningless communication is not part of the system – Meaningless communication is not part of the system – it is a it is a faultfault (faulty communication) in the context of the (faulty communication) in the context of the systemsystem

E.g., Chinese speech for non-speaker of ChineseE.g., Chinese speech for non-speaker of Chinese Note: ‘the olny iprmoetnt tihng is taht the frist and lsat Note: ‘the olny iprmoetnt tihng is taht the frist and lsat

ltteer be at the rghit pclae’ltteer be at the rghit pclae’

13

Structuring Structuring communicationcommunication

There are rules that define what There are rules that define what communication follows what communicationcommunication follows what communication

In general these rules determine which are the In general these rules determine which are the communications which another communications which another communication may referencecommunication may reference

Referencing rulesReferencing rules: what referenced : what referenced communications are needed for the production communications are needed for the production as continuation of a given communicationas continuation of a given communication

Continuation rulesContinuation rules: what communications : what communications may be produced having a given set of may be produced having a given set of communications available for referencingcommunications available for referencing

E.g., human language, animal courtship E.g., human language, animal courtship behaviourbehaviour

14

The structure of The structure of communicationscommunications

For each communication there are For each communication there are possible and not possible other possible and not possible other communications that may refer to communications that may refer to it (or which can link to it)it (or which can link to it)

E.g., grammatical rules that E.g., grammatical rules that determine which word type may determine which word type may follow other word typesfollow other word types

15

Probabilistic interpretation of Probabilistic interpretation of grammatical rulesgrammatical rules

Grammatical rules: conditional Grammatical rules: conditional probability distributions over the probability distributions over the space of possible communications, space of possible communications, which are part of the systemwhich are part of the system

E.g., human speechE.g., human speech

16

Language and grammarLanguage and grammar

Communications within a system follow Communications within a system follow the referencing and continuation rules the referencing and continuation rules expressed as conditional distributionsexpressed as conditional distributions

The sum of these reference rules forms The sum of these reference rules forms the grammar of the systemthe grammar of the system

The language of which syntax is The language of which syntax is described by the grammar is the described by the grammar is the language of the systemlanguage of the system

E.g., language of science different from E.g., language of science different from common languagecommon language

17

Identifying systems by Identifying systems by languagelanguage

Within a rich world of communications we Within a rich world of communications we can search for communications referencing can search for communications referencing other communications and which follow a other communications and which follow a well defined set of referencing and well defined set of referencing and continuation rulescontinuation rules

Separating such communications allows Separating such communications allows the identification of dense communication the identification of dense communication clusters surrounded by rare clusters surrounded by rare communicationscommunications

E.g., the system of the science of E.g., the system of the science of mathematicsmathematics

18

Time and systemsTime and systems System communications: in general they System communications: in general they

link to a pattern of other communicationslink to a pattern of other communications Special case: sequence of Special case: sequence of

communications, when communications communications, when communications can be ordered (semi-ordered) along the can be ordered (semi-ordered) along the linkslinks

Time: the order imposed by linking Time: the order imposed by linking restrictions defines the temporal structure restrictions defines the temporal structure of communications within the system of communications within the system (system time)(system time)

E.g., time in human communications, time E.g., time in human communications, time in legal communicationsin legal communications

19

What are system What are system communications about ?communications about ?

The topic of system communications is the The topic of system communications is the system itself, most generally: ‘what is the system itself, most generally: ‘what is the system and what is not the system’ – system and what is not the system’ – definition of the definition of the system’s identitysystem’s identity

They reference other communications and They reference other communications and address the issue of whether a address the issue of whether a communication is part or not of the systemcommunication is part or not of the system

E.g., legal system: legal / illegal or not E.g., legal system: legal / illegal or not addressable in these termsaddressable in these terms

20

System and environment – System and environment – 1 1

The system describes the system itselfThe system describes the system itself Another view: the system describes the Another view: the system describes the

environment by describing the environment by describing the complement of it, the systemcomplement of it, the system

The system describes the environment The system describes the environment in a complementary sense with some in a complementary sense with some closeness, but never completelycloseness, but never completely

E.g., human perception of the outside E.g., human perception of the outside worldworld

21

System and environment – System and environment – 2 2

The correctness of the system’s environment The correctness of the system’s environment description determines the ability of the description determines the ability of the system to reproduce and expandsystem to reproduce and expand

Better environment description allows the Better environment description allows the generation of communications that induce generation of communications that induce effects on the environment which favour the effects on the environment which favour the generation of more system communicationsgeneration of more system communications

E.g., antibiotic resistant and non-resistant E.g., antibiotic resistant and non-resistant bacteria, plan economy and market economybacteria, plan economy and market economy

22

Memory communicationsMemory communications

Memory communications facilitate the Memory communications facilitate the reproduction and referencing of earlier reproduction and referencing of earlier communicationscommunications

E.g. written records of a business meeting E.g. written records of a business meeting within an organisationwithin an organisation

Referencing and continuation rules can be Referencing and continuation rules can be applied using memory communicationsapplied using memory communications

Memory communications contain Memory communications contain information about the system and its information about the system and its environmentenvironment

23

StructureStructure

Structures are constraints on Structures are constraints on communicationscommunications

Structures reduce the ambiguity of Structures reduce the ambiguity of continuation and referencing rulescontinuation and referencing rules

Structures increase the ability of the Structures increase the ability of the system to reproduce and expand, if system to reproduce and expand, if they do not reduce the correctness of they do not reduce the correctness of the environment description of the the environment description of the systemsystem

24

Subsystems and Subsystems and structuresstructures

Subsystems are dense clusters of inter-Subsystems are dense clusters of inter-referencing communications within the system referencing communications within the system (denser than the system in average)(denser than the system in average)

Subsystems have their own specialist Subsystems have their own specialist language, which is a constrained version of the language, which is a constrained version of the systems language (= set of referencing and systems language (= set of referencing and continuation rules)continuation rules)

Subsystems are characterised by specific Subsystems are characterised by specific structures expressing the subsystem specific structures expressing the subsystem specific constraintsconstraints

Structures may lead to the emergence of Structures may lead to the emergence of subsystemssubsystems

25

Subsystems - examplesSubsystems - examples

Society: politics, law, economics, Society: politics, law, economics, sciencescience

Organism: neural system, Organism: neural system, circulatory system, digestive systemcirculatory system, digestive system

Subsystems specialise in some Subsystems specialise in some aspect of the system – describe this aspect of the system – describe this aspect of the system and the aspect of the system and the corresponding part of the corresponding part of the environmentenvironment

26

Information Information communicationscommunications

Communications referencing memory Communications referencing memory communications and generating new communications and generating new memory communicationsmemory communications

Information communications process Information communications process information and generate new information and generate new information within the systeminformation within the system

Information communications increase Information communications increase the system’s ability to reproduce and the system’s ability to reproduce and expand if they increase the correctness expand if they increase the correctness of the system’s environment descriptionof the system’s environment description

27

Information subsystemInformation subsystem Information subsystem: dense cluster of Information subsystem: dense cluster of

information communications within the systeminformation communications within the system E.g. nervous system within a biological E.g. nervous system within a biological

organism; management within an organisationorganism; management within an organisation The information subsystem describes a model The information subsystem describes a model

of the system (memory communications are of the system (memory communications are not necessarily complete reproductions of not necessarily complete reproductions of earlier communications) – identity modelearlier communications) – identity model

Information communications: identity Information communications: identity definition, checking and enforcementdefinition, checking and enforcement

E.g., human psyche: thinking about himself / E.g., human psyche: thinking about himself / herselfherself

28

System perceptions - System perceptions - RevisitedRevisited

The system communications are changed The system communications are changed under the effects of the environmentunder the effects of the environment

The changes are compared to the The changes are compared to the expectations based on the model/identity expectations based on the model/identity of the systemof the system

The effects of the environment are The effects of the environment are perceived (evaluated) in the perceived (evaluated) in the interpretational context of the system interpretational context of the system model/identitymodel/identity

29

System actions - RevisitedSystem actions - Revisited

The system generates new The system generates new communications about its own communications about its own model/identitymodel/identity

These communications induce effects upon These communications induce effects upon the environmentthe environment

The objective of such actions is to The objective of such actions is to reproduce and expand the system by reproduce and expand the system by continuing the analysis of other referenced continuing the analysis of other referenced communications (i.e., are they part or not communications (i.e., are they part or not of the system)of the system)

30

Action and perceptionAction and perception

Which comes first: chicken and Which comes first: chicken and egg dilemmaegg dilemma

Perceptions depend on actions Perceptions depend on actions Actions depend on perceptionsActions depend on perceptions E.g., human developmentE.g., human development

31

Double contingencyDouble contingency

Double mutual dependenceDouble mutual dependence The conditional probabilities are The conditional probabilities are

circularly conditionalcircularly conditional Communications depend on the Communications depend on the

system model, which depends on system model, which depends on communicationscommunications

32

Generating a systemGenerating a system

Double contingency is the root of the Double contingency is the root of the systemsystem

Systems emerge from a double Systems emerge from a double contingency, by questioning the identity contingency, by questioning the identity and limits of a communication cluster and limits of a communication cluster and generating further communications and generating further communications maintaining the cluster and possibly maintaining the cluster and possibly making it densermaking it denser

E.g., human psyche: ‘who am I ?’E.g., human psyche: ‘who am I ?’

33

Information subsystem and Information subsystem and double contingencydouble contingency

The emergence of the information The emergence of the information subsystem creates a new double subsystem creates a new double contingency root, helping the contingency root, helping the increased expansion of the systemincreased expansion of the system

E.g., animals with nervous systemE.g., animals with nervous system

34

Identity violationsIdentity violations Faults: communications that do fit the Faults: communications that do fit the

language of the system – not in the lexicon, language of the system – not in the lexicon, or not according to the rules of the or not according to the rules of the grammargrammar

Error: communications follow the rules of Error: communications follow the rules of the system, but it is impossible to generate the system, but it is impossible to generate a continuation for them according to the a continuation for them according to the rules of the system – the system’s rules of the system – the system’s environment description is incorrectenvironment description is incorrect

Failure: the system shrinks significantly due Failure: the system shrinks significantly due to errors and faultsto errors and faults

35

System adaptationSystem adaptation In response to identity violations the In response to identity violations the

system adapts by changing its identity – system adapts by changing its identity – i.e. by changing referencing and i.e. by changing referencing and continuation rulescontinuation rules

Adaptation starts by information subsystem Adaptation starts by information subsystem communications – identity definition, communications – identity definition, checking and enforcementchecking and enforcement

The aim of the adaptation is to prevent the The aim of the adaptation is to prevent the re-occurrence of the root of the identity re-occurrence of the root of the identity violation (e.g. fault or source of error)violation (e.g. fault or source of error)

E.g. management communications in an E.g. management communications in an adapting companyadapting company

36

Wrong adaptationWrong adaptation

Adaptation may not increase the Adaptation may not increase the correctness of the system’s correctness of the system’s environment description environment description

In case of wrong adaptation the In case of wrong adaptation the frequency of faults, errors and frequency of faults, errors and failures may increase and the failures may increase and the system may cease to existsystem may cease to exist

37

Are systems complex ?Are systems complex ?

E.g., cell, organism, society; they E.g., cell, organism, society; they are complexare complex

38

How to measure How to measure complexity ?complexity ?

Description length in some Description length in some languagelanguage

E.g., watch (mechanical system), E.g., watch (mechanical system), human body, computer programshuman body, computer programs

39

Measuring system Measuring system complexity – 1 complexity – 1

Use the system’s own languageUse the system’s own language How long is the system’s own How long is the system’s own

descriptiondescription How long is the system’s How long is the system’s

description in another supra-description in another supra-equivalent languageequivalent language

E.g., computer program, cellE.g., computer program, cell

40

Description languagesDescription languages

Two languages are equivalent if they Two languages are equivalent if they describe objects of equal complexity by describe objects of equal complexity by equal length descriptions in general equal length descriptions in general (there might be few exceptions)(there might be few exceptions)

A language is supra-equivalent A language is supra-equivalent compared to another if it can describe compared to another if it can describe the same object to the same detail in the same object to the same detail in shorter description (the other language shorter description (the other language is sub-equivalent compared to the first)is sub-equivalent compared to the first)

41

Measuring system Measuring system complexity – 2 complexity – 2

Approximation of system complexity: Approximation of system complexity: approximate description of the system approximate description of the system with some language up to some detailwith some language up to some detail

Approximate complexity: the length of Approximate complexity: the length of the approximate descriptionthe approximate description

Closeness of the approximation: how Closeness of the approximation: how much of the system behaviour is not much of the system behaviour is not described by the approximate described by the approximate descriptiondescription

42

The complexity of the The complexity of the environmentenvironment

By definition the environment is By definition the environment is infinitely complexinfinitely complex

The environment is the outside of The environment is the outside of the system, if it could be described the system, if it could be described completely it would become part of completely it would become part of the systemthe system

43

Systems surviving in an Systems surviving in an environmentenvironment

A system survives: reproduces and A system survives: reproduces and expands if it’s description of the expands if it’s description of the environment (of itself) captures a environment (of itself) captures a good part of the environment to good part of the environment to generate the appropriate actions generate the appropriate actions to maintain and expand itselfto maintain and expand itself

E.g., cell, organismE.g., cell, organism

44

Complex systemsComplex systems

As the environment is infinitely As the environment is infinitely complex, systems which survive in complex, systems which survive in the environment are very complex the environment are very complex

The complexity of a system is The complexity of a system is reflected by the part of the reflected by the part of the environment that can be described environment that can be described in the system language (in in the system language (in complementary terms)complementary terms)

45

Summary – 1 Summary – 1

MeaningMeaning LanguageLanguage System languageSystem language MemoryMemory StructureStructure

46

Summary – 2 Summary – 2

SubsystemsSubsystems Information subsystemsInformation subsystems System identitySystem identity Identity violation and adaptationIdentity violation and adaptation Double contingencyDouble contingency Complexity Complexity

47

Q&A – 1 Q&A – 1

1.1. Is it rue that the sequence of letter Is it rue that the sequence of letter ‘qraywtta’ conveys some meaning for ‘qraywtta’ conveys some meaning for English speakers ?English speakers ?

2.2. What about the sequence ‘strawberry’ ?What about the sequence ‘strawberry’ ?

3.3. Is it true that the courtship behaviour of Is it true that the courtship behaviour of animals has a grammar ?animals has a grammar ?

4.4. Is it true that flat owners form a system Is it true that flat owners form a system that is identifiable by its specific that is identifiable by its specific language ?language ?

48

Q&A – 2 Q&A – 2

5. What about the goths ?5. What about the goths ?6. Is it true that the structure of a system 6. Is it true that the structure of a system

language can be seen as a rules of language can be seen as a rules of restrictions on linking communications restrictions on linking communications by references ?by references ?

7. Is it true that politicians communicate 7. Is it true that politicians communicate mostly about the welfare of people ?mostly about the welfare of people ?

8. Is it true that the nervous system is the 8. Is it true that the nervous system is the specialist subsystem of the organism specialist subsystem of the organism dealing with information processing dealing with information processing within the organism ?within the organism ?