Full Text 27

7/28/2019 Full Text 27

1/18

Biosemiotics (2009) 2:303320DOI 10.1007/s12304-009-9062-4

ORIGINAL PAPER

Applying Semiotics and Information Theoryto Biology: A Critical Comparison

Grard Battail

Received: 15 July 2008 / Accepted: 10 October 2008 /

Published online: 2 October 2009 Springer Science + Business Media B.V. 2009

Abstract Since the beginning of the XX-th century, it became increasinglyevident that information, besides matter and energy, is a major actor in thelife processes. Moreover, communication of information has been recognizedas differentiating living things from inanimate ones, hence as specic to the lifeprocesses. Therefore the sciences of matter and energy, chemistry and physics,do not sufce to deal with life processes. Biology should also rely on sciencesof information. A majority of biologists, however, did not change their mindand continued to describe life in terms of chemistry and physics. They merelyborrowed some vocabulary from the information sciences. The rst scienceof information available to biological applications, semiotics, appeared at theend of the XIX-th century. It is a qualitative and descriptive science whichstemmed from efforts of linguists and philosophers to understand the humanlanguage and is thus mainly concerned with semantics . Applying semiotics tobiology resulted in todays Biosemiotics . Independently, an explosive expan-sion of communication engineering began in the second half of the XX-th

century. Besides tremendous progresses in hardware technology, it was madepossible by the onset of a science of literal communication: Information Theory(Shannon, Bell Syst Tech J 27:379457, 623656, 1948). Literal communicationconsists of faithfully transporting a message from a place to another, or froman instant to another. Because the meaning of a message does not matter for itstransportation, information theory ignores semantics. This restriction enables

Parts of this material have been presented at the 8-th Biosemiotics Gathering, 2328 June2008, Syros, Greece. Submitted to Biosemiotics .

The author has retired from ENST, Paris, France.

G. Battail ( B )La Chanatte, le Guimand, 26120 Chabeuil, Francee-mail: [email protected]

7/28/2019 Full Text 27

2/18

304 G. Battail

dening information as a measurable quantity on which a mathematical theoryof communication is founded. Although lacking implementation means atits beginning, information theory became later very successful for designingcommunication means. Modern ones, like mobile phones, can be thought of as experimentally proving the relevance and accuracy of information theorysince their design and operation heavily rely on it. Information theory isplainly relevant to biological functions which involve literal communication,especially heredity. This paper is intended to compare the two approaches. Itshows that, besides obvious differences, they have some points in common: forinstance, the quantitative measurement of information obeys Peirces triadicparadigm. They also can mutually enlighten each other. Using informationtheory, which is closer to the basic communication mechanisms, may appearas a preliminary step prior to more elaborated investigations. Criticizing

genetics from outside, information theory furthermore reveals that the abilityof the template-replication paradigm to faithfully conserve genomes is but aprejudice. Heredity actually demands error-correcting means which imposesevere constraints to the living world and must be recognized as biologicalfacts.

Keywords Biosemiotics Error-correcting codes Heredity Information theory Semantics

Introduction

Accounting for the exchange of signals of all kind (chemical, electrical, optical,acoustical, . . . ) which occurs as an integral part of the life processes, at all scalesfrom the molecules to the ecosystems, biosemiotics happily complementstraditional biology. It is intended to fully take into account the communicationof information in the living world, which is more and more recognized as

differentiating it from the inanimate world, hence as fundamental in the lifeprocesses. Communication plays indeed in the living world an essential rlewhich moreover is specic to life. It is not reducible to physics and chemistrybut what is communicated information is an entity of its own. Biology must thus integrate the science of information. But there is no unied science of information. Two such sciences are available: semiotics and information theory .The former and oldest is at the origin of biosemiotics which can already gathera number of experts. The second one has been extraordinarily successful inengineering but very few people seriously try to apply it to life. The great

pioneer here is Yockey ( 1992, 2005).Biosemiotics borrows its paradigms from the communication between hu-mans and thus incurs the risk of anthropocentrism. The founding fathersof semiotics lived well before the explosive expansion of communicationengineering in the second half of the XX-th century. The idea that functions of abstract nature actually need a physical implementation was foreign to them.

7/28/2019 Full Text 27

3/18

Applying Semiotics and Information Theory to Biology: A Critical Comparison 305

The only instances of such an implementation they thought of were mental,i.e., involving the human brain, an organ of immense complexity which, acentury later, is still poorly understood despite much works and progresses.The brain mechanisms were completely unknown when semiotics arised soit could only develop as an abstract, speculative discipline deprived of thesupport of experiments, except for mental ones.

By now, communication (and even communicating) machines have literallyinvaded our lives. The progress of communication technology was madepossible not only by tremendous advances in the physical hardware, but alsobecause information theory, which originated in a reection about commu-nication technology, provided a sound theoretical basis for the design andanalysis of communication systems. Information theory lays emphasis on literal communication and quantitative methods. Its development was made possible

thanks to a bold a priori position: to discard semantics . In the very rst page of the paper which founds information theory (Shannon 1948), Shannon wrote:

The fundamental problem of communication is that of reproducing atone point either exactly or approximately a message selected at anotherpoint. Frequently the messages have meaning ; that is they refer to or arecorrelated according to some system with certain physical or conceptualentities. These semantic aspects of communication are irrelevant to theengineering problem. The signicant aspect is that the actual message is

one selected from a set of possible messages. The system must be designedto operate for each possible selection, not just the one which will actuallybe chosen since this is unknown at the time of design. (Shannons italics.)

Just like a messenger has not to know about the message he/her carries,a communication system ignores meaning and should operate regardless of the particular message it has to transmit. Replacing in the above quotationthe word point which refers to a location in space with the word instantmoreover extends its relevance to communication in time, hence to heredity.

Two Contrasting Sciences

It is hard to imagine that two sciences of communication could be based onsuch opposite postulates. For semiotics, semantics is the essence of communi-cation and thus its main object; if not overlooked, literal communication isa trivial matter which deserves no special interest. For information theory,semantics must be discarded as irrelevant to the engineering problem of

communication, which merely consists of making the transmitted messageavailable to its destination. It is a necessary step in any communicationhowever not the ultimate one. Information theory may thus be consideredas the prerequisite to further renements accounting for semantics. It maybe thought of as an unavoidable intermediate in the path from conventionalbiology towards biosemiotics.

7/28/2019 Full Text 27

4/18

306 G. Battail

Table 1 Main features of semiotics and information theory

Semiotics Information theoryBeginning End of XIX-th century Shannon ( 1948)

de Saussure, Peirce

Originated in Philosophy and Linguistics EngineeringConcerned with Semantic communication Literal communicationDifculty Intrinsically difcult as facing Basically easier as discarding

semantic problems semanticsStyle Descriptive and qualitative Mathematical and quantitativeImpact on the Foreign to implementation Useful in designing devices,

physical world systems and processesKnowledge of borders Found absolute limits for

possible communication

The main features of semiotics and information theory have been gatheredin Table 1. Each of its lines points out a feature about which they disagree andthe remainder of this paper will elaborate on these contrasts, trying to showwhat benets information theory can bring to biology.

What Engineering can Bring to Biology

Reviewing in Benner ( 2008) a book by Regis ( 2008), Steven Benner wrote:

Because building something requires a deep understanding of its parts[and of their mutual relationship], synthesis also stops scientists fromfooling themselves. Data are rarely collected neutrally during analysesby researchers, who may discard some, believing the data to be wrong if they do not meet their expectations. Synthesis helps manage this problem.Failures in understanding mean that the synthesis fails, forcing discoveryand paradigm change in ways that analysis does not. (The phrase inbrackets has been added by me.)

Synthesis being the engineers job, this remark is an excellent plea for a close

collaboration of biologists and engineers. It is originally intended to geneticengineering but actually applies to any instance where nature and engineersare faced with the same problems. It puts in the forefront the necessaryimplementation of biological functions. Indeed, assuming the existence of somebiological function without caring about how it is implemented pertains towishful thinking. The engineering approach advocated in the above quotationshould avoid it. Besides a renewed understanding of biological facts, anotherbenet of an engineering approach regards methodology: it makes possiblequantitative assessments.

Communication engineering benets from the theoretical framework of information theory. Literal communication of sequences of symbols (literalmeaning that semantics is ignored) is actually a mathematical problem, andinformation theory is just that branch of mathematics which deals with it.

Information theory can bring to biology its concepts and methods as wellas its results. Maybe its most important concept is that of channel capacity ,

7/28/2019 Full Text 27

5/18


proven to set an impassable limit to any communication. Information theoryactually proves that communication without errors (more precisely, with anarbitrarily small error rate) is possible over a channel despite the errors whichaffect it, provided the information rate is less than the channel capacity, aquantity which decreases as the channel error rate increases. However, thevery means which enable errorless communication hinder any communica-tion at all beyond the channel capacity. Both the possibility of errorlesscommunication below the capacity and its impossibility above it, althoughrather counterintuitive, are fully conrmed by the engineers experience,besides being theoretically proven.

As an engineering discipline, information theory had a tremendous impacton the communication techniques. It established the limits of what is possible,and the engineers strove for approaching them. We briey summarize the

parallel development of information theory and communication engineeringin the next section.

Information Theory and Communication Engineering

Two events of capital importance for the future of communication engineeringoccurred simultaneously at the same place: in 1948, at the Bell TelephoneLaboratories, Claude Shannon published A mathematical theory of commu-nication (Shannon 1948); and John Bardeen, William Shockley and Walter

Brattain invented the transistor . The technological developments based onthe second event, i.e., the semi-conductor technology, provided means toimplement solutions to communication problems having their origin in therst. It turns out that the information-theoretic solutions to communicationproblems are the more efcient, the more complex. The progress of semi-conductor technology resulted in devices becoming at the same time more andmore tiny and more and more complex. By now, 60 years after the transistorinvention, a silicon chip of a few square centimetres can bear about a billiontransistors. The tremendous evolution of semi-conductor technology towards

increasing complexity perfectly tted the needs of communication engineeringfor implementing solutions inspired by information theory. Information theoryespecially led to the development of very sophisticated error-correcting codeswhich reliable and inexpensive devices can by now implement. They actuallyinvaded our daily life: computer memories, mobile phones, CD, DVD, digitaltelevision . . . However, they remain invisible and very few people are awareof the enormous complexity which subtends electronic objects of daily use.With their trend towards complexity and small size, electronic devices tendto mimic biological devices. Just like we are unaware of the physiological

processes which keep us alive, we are less and less conscious of the complexityof the electronic objects which we routinely use. Most of us completely ignorehow they work. Moreover, explaining their operation often needs advancedconcepts, mainly borrowed from information theory.

Information theory originated in Shannons paper (Shannon 1948). It gaverise to a new science but, according to an approach almost unique in his-

7/28/2019 Full Text 27

6/18

308 G. Battail

tory, this paper also constitutes a complete treatise of the nascient science.Theoretical developments were nevertheless needed to conrm Shannonsstatements, especially as regards the mathematical rigour of his proofs, butlittle was left to Shannons successors for deepening and expanding informa-tion theory. One of the most important theoretical events in this respect hasbeen the introduction of the algorithmic information theory , Kolmogorov andChaitin around 1965, see Chaitin ( 2005), which, at variance with Shannons,does not rely on probabilities. If Shannon left comparatively little to be doneon the theoretical side, his papers prompted countless, entirely unexpected ap-plications in the eld of source- and channel coding . Source coding consists of replacing an initial message by a shorter but fully equivalent one. Channel cod-ing aims at protecting an initial message against transmission errors; it neces-sarily introduces redundancy, i.e., replaces the original message by a longer

one. Then within certain limits errors in the encoded message do not hinderrecovering the initial one. As regards source coding, the Huffman algorithmasymptotically achieved the theoretical limit stated by information theory,namely, the source entropy, as early as 1952. Other efcient source codingalgorithms were found later (arithmetic coding, Lempel-Ziv algorithm, . . . ).In sharp contrast, while information theory also stated the limit of what ispossible as regards channel coding, namely, the channel capacity , no practicalmeans to closely approach it were found during decades although it has beenperceived as a challenge by thousands of mathematicians and engineers and

thus prompted intense researches. The goal was not achieved earlier than1993 when the invention of turbocodes by Berrou and Glavieux (Berrou etal. 1993; Berrou and Glavieux 1996; Guizzo 2004) provided practical meansto communicate at information rates close to the channel capacity, henceexperimentally proving the relevance of Shannons channel coding theorem.

Is Literal Communication a Trivial Problem?

Is literal communication so trivial a problem? As dened in Shannons quo-

tation of Introduction , it simply consists of making the transmitted messageavailable to its destination. The message is generated at a distance (in spaceand/or time) from the destination so it needs be transported. Transporting themessage generally needs its transformation by source- and/or channel coding.It can actually be transformed into an innity of equivalent messages whichpossibly differ as regards their physical support, the size of the alphabet in use,and coding operated on the original message. Hence an information must beseen as an equivalence class with respect to all such transformations.

As an example, the sequence of Latin letters:

Information theory discards semantics (1)and the binary sequence1001001 1101110 1100110 1101111 1110010 1101101 1100001 1110100 11010011101111 1101110 0100000 1110100 1101000 1100101 1101111 1110010 11110010100000 1100100 1101001 1110011 1100011 1100001 1110010 1100100 1110011

7/28/2019 Full Text 27

7/18


0100000 1110011 1100101 1101101 1100001 1101110 1110100 1101001 11000111110011share the same information , since the latter just resulted from transformingsequence (1) using the ASCII (American Standard Code for InformationInterchange) code which is currently used in computer memories. Each Latinletter is replaced by a 7-bit 1 word according to a one-to-one correspondence.

The binary sequence10010011 11011101 11001100 11011110 11100100 11011011 11000011 1110100011010010 11011110 11011101 01000001 11101000 11010001 11001010 1101111011100100 11110011 01000001 11001001 11010010 11100111 11000110 1100001111100100 11001001 11100111 01000001 11100111 11001010 11011011 1100001111011101 11101000 11010010 11000110 11100111also bears the same information as sentence (1) since an 8-th bit has been

appended to each of the 7-bit words of the previous binary sequence, equalto the sum modulo 2 of its bits thus making the total number of 1s even. Thismay be thought of as a rudimentary means of error control: if an error affects asymbol in a 8-bit word, the number of 1s becomes odd so counting the 1s ineach word enables detecting a single-symbol error. Of course, the rst binarysequence could be transformed by sophisticated error-correcting codes into anequivalent one made resilient to errors (up to a limit) and bearing again thesame information.

As a counterexample, the sentence:

La thorie de linformation exclut la smantique (2)

is a French translation of the English sequence (1). Although it looks close toit, articles and a preposition have been appended to comply with the Frenchgrammar and the word order is different, so sentence (2) does not bear thesame information as (1). However, both sentences share the same meaning .

These remarks will be further developped in Information and its Relation-ship to Semantics .

On Semantic Communication

Ignoring semantics, information theory only deals with (literal) informationsas just dened. Exclusion of semantics is a strength, by no means a weakness,of information theory. Semantics depends on interpretation rules, which them-selves belong to the semantic eld. Therefore, a text written in any language,say English, can state that it changes the meaning of the words. Imagine atext which tells that, from now on, the word table will be used in order tomean point and the word chair to mean straight line. Then, the nonsensical

sentence one and only one chair passes through two tables becomes an axiomof Euclidean geometry (interestingly, this example is borrowed from the great

1We systematically use the acronym bit to designate a binary digit and the word shannon,abbreviated as Sh, for the binary unit of information, originally named bit by Shannon.

7/28/2019 Full Text 27

8/18

310 G. Battail

mathematician David Hilbert, not from a linguist). Similarly, during WorldWar II, the BBC broadcast apparently nonsensical messages intended to theResistance ghters, to whom they were indeed very meaningful. Such remarkssufce to make the concept of meaning extremely difcult to use in a scienticcontext. The information-theoretic concept of information does not sufferthe same basic weakness, although it restricts of course the semantic contentof the word information. Discarding semantics from the very beginning,information theory has cut the Gordian knot.

There is, however, a scientic domain where using the concept of meaningsuffers less difculties and this exception, interestingly, is biology. We mayindeed reasonably assume that nature does not intend to fool researchers whilesuch an intent cannot be excluded in human communication. 2 Due to theassumed non-malignancy of nature, a kind of comparatively crude semiotics

can be successfully used in biology, free from the endless semantic subtletiesthat characterize human communication and interaction. For instance, the ge-netic code 3 denitely establishes a correspondence between the 3-nucleotidecodons of messenger RNA and the amino-acids in proteins so we may sayunambiguously, e.g., that the meaning of the codon UAC is the amino-acidthyrosine.

The remark that taking semantics into account increases the difculty of communication problems, even if malignancy can be excluded, clearly estab-lishes a hierarchy of difculty between semiotics and information theory. The

former is clearly more difcult than the latter. The logical order for applyingthese sciences to biology would thus be to begin with the latter, which isbasically simpler. Then, but only then, could problems involving semantics beattacked. Unfortunately, the historical order has been the reverse. In cruderwords, biology has put the cart before the horse.

Information and its Relationship to Semantics

Shannon and the pioneers of information theory had an empirical approachsince they did not attempt dening information, nor explicating its connectionwith semantics. They just proposed means for its quantitative measurement.We try here rather navely to dene information, hopefully shedding somelight on the relationship of information and semantics. These remarks arepersonal and do not express any consensus among information theorists.

Let us consider a digital message, i.e., a sequence of symbols from somenite-size alphabet. Such a message is a mathematical abstraction which needsa physical support for having any interaction with the real world, and especially

2Philip Henry Gosse (1816,1888) even attributed this intent to God in his book Omphalos ,published in 1857, aimed at conciliating the data of geology with the Biblical account of creation.3In the phrase genetic code which appeared in the sixties, the word code has been given ameaning rather foreign to its earlier use in information theory, that of a correspondence rulebetween objects of different nature, i.e., nucleotides and amino-acids.

7/28/2019 Full Text 27

9/18


for being communicated. The physical support of a message can take a varietyof forms, either material (e.g., ink deposits on a sheet of paper according toconventional patterns, i.e., letters, or holes punched in a card, or local statesof magnetization of some ferromagnetic medium, or shallow tiny holes ina compact disk (CD), ...), or consisting of the variation of some physicalquantity as a function of time (e.g., air pressure, or electrical current, or electro-magnetic eld, . . . ). The material supports are actually used for recording, i.e.,communication through time consisting of writing a message to be read later,while the variation in time of a physical quantity can be propagated as a wavehence enables communication through space. Regardless of its support, eachof the alphabet symbols needs only be unambiguously distinguishable from theother ones.

A same message can be supported by different physical media which more-

over can be converted from one to another. For instance, the message recordedon a computer memory or a CD can be converted into an acoustic wave(i.e., a sound), or emitted as an electromagnetic wave. Similarly, the alphabetsize can be changed: a musical record in a computer memory uses the binaryalphabet, but a large-size alphabet is needed for converting it into an audibleacoustical signal. The message itself can moreover be changed so as to improveits characteristics as regards some desired property. For instance, it may becompressed so that its recording needs less memory space (this is source codingas alluded to above), or on the contrary expanded by a redundant encoding

making it resist transmission errors (this is channel coding).A message can thus exist into a variety of equivalent forms, depending onits possible encoding, alphabet size, and physical support. We refer to theunderlying entity which is common to all these forms as an information . Wemay thus dene an information as the equivalence class of all the messageswhich can be converted to each other by changing their encoding, alphabetsize, or physical support. Each of these messages will be said to bear thisinformation. The equivalence class associated with an information clearlycontains innitely many elements. Any of these messages is a representative

of this class. An information is thus an abstract object which manifests itself inthe physical world by any of its representatives, or realizations .The following question immediately arises: given some alphabet with an

arbitrary physical support, is there a minimal-length realization of a giveninformation? The simplest alphabet, i.e., the binary one, is a natural choice.The problem becomes whether a minimal-length binary message exists withinthe equivalence class associated with the given information. Shannons infor-mation theory asserts this existence when the given representative is a messagegenerated by a stationary probabilistic source. The algorithmic informationtheory extends this statement to any message that a universal computer cangenerate. Well refer to the minimal binary realization of an informationas its information message . The fundamental theorem of source coding of Shannons information theory states that the average length of the informationmessage equals the length of the original message times the source entropyexpressed using binary information units, i.e., shannons. In the algorithmic

7/28/2019 Full Text 27

10/18

312 G. Battail

information theory, the length of the information message is used for deningthe algorithmic complexity associated with the given information. The rstcase is much more restrictive than the second one, which may be consideredas general. However, the source stationarity enables effectively estimatingthe probabilities of the sequences it generates, hence its entropy, by makingfrequency measurements. In the second case, on the contrary, the existenceof a minimal-length realization is mathematically proven, but the algorithmiccomplexity is actually an uncomputable quantity which thus can generally notbe evaluated.

Given a stationary source of entropy per symbol H shannons, the funda-mental theorem of source coding states that any n-symbol message generatedby this source can be transformed by source coding into a binary messageof average length at least = nH bits. This minimal-length realization of an

information generated by a probabilistic source is its information message. Itresults from optimal source coding, which entails that its bits are probabilisti-cally independent and equiprobable.

The set of all binary messages of length can be represented by a tree likethat of Fig. 1 with each of its branches labelled with a bit according to someconvention. Let us interpret the i-th bit of the information message as theanswer to a dichotomic question (i.e., answerable by yes-or-no), 0 meaning

Fig. 1 Binary tree forrepresenting all binarysequences of length 4. Anascending branch representsthe bit 0 and a descending onethe bit 1

Root

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

7/28/2019 Full Text 27

11/18


for instance yes and 1 meaning no. Any information message of lengthmay thus be interpreted as one of the 2 paths of a binary tree of lengthtaken from the root to the leaves, the choice of a branch at any fork beingmade at random with probability 1/2. A path in this tree, i.e., an informationmessage of length , can be interpreted as an integer i , 0 i 2 1. The ques-tions associated with the successive bits of the information message may forinstance be those needed for identifying the species to which some given livingbeing belongs. Provided the set of species is ordered according to a binaryhierarchical taxonomy, using = nH properly chosen successive dichotomicquestions enables distinguishing from each other 2nH = (2 H )n species. Theentropy H of the source then measures the ability of the messages it generatesto discriminate among objects of some outer world, hence to bear some kindof semantics. The corresponding information quantity is simply the number of

binary digits which are needed to represent it.Establishing a correspondence between each bit of the information message

and a dichotomic question makes it eventually identify or represent an objectwhich belongs to some outer world, like living beings as in the above exampleof taxonomy, making possible to distinguish between them. Then informationin the above meaning is given a semantic content according to an externalconvention which we may refer to as its meaning .

This is a rather crude kind of semantics, apparently restricted to represent-ing material objects which can be ordered in a tree-like fashion. However, the

possible semantic content can be widely extended if we notice that: Describing an outer reality by a binary message is not limited to answering

dichotomic questions. Data of various other kind can also be represented.For instance, grouping k bits of the information message into a blockmay be used to specify that the value of some parameter is one of 2kpredetermined levels. If k is large enough, this amounts to approximatelyspecify the value of a continuously varying parameter. This is for instancecurrently used in telephony to represent instantaneous values of the speechsignal (referred to as samples) by a sequence of bits, a process referred toas pulse code modulation (PCM). For frequent enough samples (8 kHz,i.e., a sample every 125 s) and k as low as 8 (hence 28 = 256 levels), asufcient quality of speech transmission is achieved.

Moreover, a relation between material objects can be represented by thesame means as the objects themselves, then opening the semantic eld toabstract as well as material objects.

Information theory, either Shannons or algorithmic, uses the length of the information message as a quantitative measure of information. Since an

information message of length enables distinguishing 2 different objects, isa logarithmic measure of the discriminating ability of the information message,regardless of the distinguished objects, a matter of semantics. We may thusunderstand a quantity of information as the number of semantic instances itcan represent, or as the number of dimensions of some space which representssemantic objects. It should be kept in mind that the information quantity is by

7/28/2019 Full Text 27

12/18

314 G. Battail

no means an exhaustive description of an information, just like the mass is onlyone of the many attributes that a material object possesses.

Notice that in the information-theoretic literature, the word informationis most often used to mean measure or quantity of information. For instance,mutual information refers to the quantity of information that the output of a channel provides as regards its input. The word mutual expresses that it isalso the quantity of information that the input of a channel provides about itsoutput. The capacity of a channel is the largest possible mutual informationbetween its input and its output.

Some Remarks About Information Theory

Literal communication being basically simpler, information theory developedas a mathematical and quantitative science. However, the necessity of imple-menting the functions of communication revealed unexpected difculties. Thispoint will be illustrated by comparing quotations from Marcello Barbieri andClaude Shannon.

Barbier i (2008) denes semiosis as the production of signs, while Shannon,in the text already quoted in the introduction, writes that the fundamentalproblem of communication is that of re producing at one point [. . . ] a messageselected at another point. Shannons formulation implies that the difculty of engineering communication mainly lays at the receiving end. Indeed, since theselected message is unknown at the receiving end, the choice which has beenmade at the transmitting end must be inferred . Producing signs is, in a sense,trivial. Inferring what signs have been produced implies that the receivingend deals with the possibly sent signs as chance events, which entails that itneeds knowing the repertoire of signs which can occur. Moreover, the intendeddestination of the produced signs seldom (actually never) escapes outer inu-ences. The signs which are produced are thus not perceived in isolation, butin the presence of noise , this word designating the collective result of external

inuences which can only be dealt with as random . The receiving process isthus basically probabilistic, hence has a nonzero probability of failure or error.Reection about implementation reveals here an overlooked difculty.

Measuring Information Conforms to Peirces Triadic Paradigm

The necessity for the receiver to know the repertoire of signs in use resultsin the measure of information conforming to Peirces triadic paradigm. Let usconsider a set M of M randomly occurring events or messages. Let pm denotethe probability that the particular message m occurs. Since a message of the setM must occur, we have

M

m= 1

pm = 1. (1)

7/28/2019 Full Text 27

13/18


Information theory measures the information quantity brought by theoccurrence of the particular message m by

hm = log (1/ pm ) = log pm , (2)

which is positive (more precisely, nonnegative) since pm 1. Then the moreimprobable is m , the more information it bears.

The entropy associated with M is the information quantity brought in theaverage by the occurrence of one of its elements, namely,

H =M

m= 1

pm hm = M

m= 1

pm log pm . (3)

This information measure is relevant to the set of messages M as a whole, not

to any particular message which belongs to it.Now consider the information quantity hm associated with a single message

m according to ( 2). It does not depend on the message m itself, but only on theprobability pm of its occurrence. That the probabilities of all the messages in Msum up to 1 according to ( 1) entails that it is not possible to change one of theseprobabilities without changing others. Hence the information measure thatinformation theory associates with a single event m according to ( 2) dependson the context in which it occurs, namely, the probabilities of the events inthe set M to which it belongs: in a sense, the information borne by the

occurrence of a message depends on the probabilities of the messages whichwere not selected as well as that of the single one which was. Measuring theinformation of a single event thus needs an interpretation in terms of its context .

No Denite Information is Borne by a Single Sequence

As an important consequence, it is meaningless to refer to the informationborne by a sequence since a single sequence can belong to an innity of different contexts. For instance, the 8-digit sequence 10101010 bears at most

8 binary information units (shannons) if its digits are assumed to belong tothe binary alphabet. However, 0 and 1 are also decimal digits as belonging tothe set {0, 1, 2, ..., 9} (they are actually digits in a numeration system to anarbitrary base b , b 2). If the sequence 10101010 is interpreted as decimal, itbears log 2 10 times more information, i.e., approximately 26.6 Sh at most.

An attempt to prove the concept of intelligent design allegedly usinginformation-theoretic arguments, in the book by Pullen ( 2005), is wrong be-cause single sequences are assumed to bear absolute quantities of information.Incidentally, a major argument in favour of intelligent design is that the

probability of a single-nucleotide mutation being of about 10 8

per generation,a mutation involving two single-nucleotide mutations, assumed to entail asignicant phenotypic difference, has a per generation probability of (10 8)2 =10 16 , hence is practically impossible. This would be true only if the two single-nucleotide mutations were independent events. This is not true, however, if both nucleotides belong to a word of a genomic error-correcting code. It is

7/28/2019 Full Text 27

14/18

316 G. Battail

this codeword as a whole which is erroneously chosen if a regeneration erroroccurs and its symbols are strongly correlated. Ironically, the upholders of mainstream biology rightfully reply that the two single-nucleotide mutationsare not independent events but simultaneously ignore the necessity of agenomic error-correcting code (to be recalled later) which is precisely thereason why they are correlated.

Difculties are Mainly Found at the Receiving End

Realizing the unequal processing difculty at the transmitting and receivingends illustrates one of the benets of an engineering approach pointed out byBenner as quoted in What Engineering can Bring to Biology : implementa-tion questions how certain functions are performed and reveals unexpected

difculties. In the biological literature, the complexity of processes which takeplace at the receiving end is typically overlooked. This is especially true forrecognition processes, the importance of which is capital in the living world.Humans, as most living beings, are very efcient in the tasks of recognition. It isonly when engineers tried to design machines able to perform such tasks that itwas realized how they were difcult. The laymen think of them as natural andcompletely overlook the needed complexity of the mechanisms which performrecognition. As yet, the machines designed to this end, however sophisticated,remain signicantly less efcient than those that nature implemented. More-

over, the best of them use articial neural networks which need some learningprocess, i.e., mimic natural devices and processes.

Heredity as Literal Communication

Heredity is a problem of literal communication since it consists of communicat-ing over time some message, the genome . The faithful communication of ge-nomes is of capital importance for the living world as a whole.

Let us consider the two functions of the genome, as illustrated in Fig. 2.As replicating itself, a genome provides (1) another identical genome, whereidentical means that it is conserved at the geological timescale. A genomealso (2) instructs the construction of a phenotype. Performing (1) is a problemof purely literal communication, hence fully relevant to information theory.We shall show below the benets that applying information theory to thisproblem can provide to genetics. Well especially show that, contrary to thecurrent belief, the template-replication paradigm does not sufce to solveit. Performing (2) involves semantics and thus escapes the competence of

information theory, so well let it aside. Notice that problem (2) is dealt with ina vast majority of papers. This may be due to the fact that we are phenotypes(with dormant genomes inside them) so we are more or less consciouslyaffected with phenotypocentrism. Indeed, the two functions are absolutelynecessary, but (1) plays the rle of the egg in the chicken-and-egg dilemma,which may be thought of as the more basic.

7/28/2019 Full Text 27

15/18


GENOME

PHENOTYPE

IDENTICALGENOME

1

2

Fig. 2 What a genome generates

Conserving a message using any kind of memory, for instance DNA if itis a genome, is a problem of literal communication (over time) hence fullyrelevant to information theory. As a communication channel, any permanentmemory has a capacity: an upper bound of which can easily be computed

(Battail 2006b). It turns out that, except if the error rate is exactly 0, thiscapacity vanishes exponentially fast. In adamant words, no static memory ispermanent. Since genetic mutations occur with nonzero probability, DNAalone cannot conserve the genome. This sufces to refute the template-replication paradigm.

The trouble with template-replication is that copying reproduces the erro-neous symbols as well as the correct ones. The genome should not be copied,but made resilient to errors by means of an error-correcting code. Then, pro-vided the cumulated number of errors remains less than some threshold, the

genome can be regenerated . Regeneration is intended to rewrite the genomicmessage in such a way that:

the rewritten message strictly satises the constraints which dene thegenomic code;

and is the closest to the original genomic message.

Contrary to replication, regeneration does not result in a message faithfulto the original one. It is however faithful to the genomic code seen as a set of constraints.

Then, the casual errors which may affect the genome are corrected, exceptif these errors are as numerous as to exceed the error-correcting ability of the code. If such a regeneration error occurs, it results in a widely differentgenome. For a long enough and properly designed code, and a short enoughinterval between regenerations, the probability of a regeneration error isextremely small. The crucial importance of conserving genomes necessarily

7/28/2019 Full Text 27

16/18

318 G. Battail

led natural selection to retain a good enough genomic code and a short enoughtime interval between successive regenerations.

Regeneration in itself does not sufce to ensure the survival of a species: thenumber of its members should tend to increase, so a genome must be replicatedonce it has been regenerated. As regards its implementation, regeneration ismuch more costly than replication in terms of processing complexity.

We thus assume that:

a genomic error-correcting code exists (main hypothesis); this genomic code unequally protects the data, since some of the oldest

are the most faithfully conserved. A system of nested component codes isproposed as performing so (subsidiary hypothesis).

Assuming the above main and subsidiary hypotheses to be true, as we didrst in Battai l (1997), explains very basic features of the living world, e.g.:

that nature proceeds with successive generations (the number of cumu-lated errors should not exceed the correcting ability of the code);

the existence of discrete species directly results from the main hypothesis.Moreover, the subsidiary hypothesis entails that they can be orderedaccording to a hierarchical taxonomy which coincides with phylogeny;

the trend of evolution towards increasing complexity, as a result of naturalselection operating on the genomic error-correcting codes and favouringefcient codes, which need be longer to be more efcient as shown byinformation theory.

These topics were dealt with at length in previous works (Battail 2006a, c,2007, 2008).

The above remarks illustrate the benet that an external discipline canprovide to biology by bringing ideas already foreign to it. This may helpdetecting and correcting prejudices and even possible misconceptions whichremain unquestioned within its own framework. If a discipline tries to solve

problems as they are posed within the conventional framework of anotherdiscipline, prejudices and misconceptions can be inherited within the problemstatements. Then the main advantage of interdisciplinarity, criticism from theoutside, is lost. It turns out that the semiotic approach to biology has not hadthe same critical rle as information theory: its attention has been restricted tothe processes by which a genome generates a phenotype . Semantics is centralin this problem. Being mainly concerned with semantics, semiotics naturallyinherited this prejudice from molecular biology.

The goal of synthetic biology, articial life, is very ambitious, Promethean

indeed (Regis 2008). Much more modestly, having an engineering look at func-tions basic to life can provide biology with the claimed benets of synthesis,namely, to quote again Benner, forcing discovery and paradigm change. Asfully relevant to information theory, heredity is very interesting in this respect.The verdict of information theory is nal: the template-replication paradigmhas to be replaced by that of genomic error-correcting code.

7/28/2019 Full Text 27

17/18


Conclusion

Dawkins wrote in The selsh gene (Dawkins 1976), 1976:

We do not know how accurately the original replicator molecules madetheir copies. Their modern descendants, the DNA molecules, are aston-ishingly faithful compared with the most high-delity human copyingprocesses.

We have shown in earlier works and repeated above that copying is not thefunction that DNA molecules should implement, which instead must actuallyinvolve error-correcting means enabling their regeneration. Doing so, weactually complied with Dawkinss further remark (in The Blind Watchmaker ,(Dawkins 1986), 1986):

If you want to understand life, dont think about vibrant, throbbing gelsand oozes, think about information technology.

However, the semi-conductor hardware is quite foreign to the enzyme-catalyzed reactions which occur in the cell. It is not at the level of implemen-tation means that information technology resembles life, but as regards thealgorithms which are implemented. Thus, we can fully agree with the abovequotation only if theory is substituted for technology.

Indeed, biology must hear the lessons of information theory. A collaboration

between biologists and information theorists will be highly benecial forboth. Maybe what mostly lacks for establishing such a collaboration is thatinformation theory be properly popularized .

References

Barbieri, M. (2008). What is biosemiotics? Biosemiotics, 1 , 13.Battail, G. (1997). Does information theory explain biological evolution? Europhysics Letters,

40(3), 343348.Battail, G. (2006a) Information theory and error-correcting codes in genetics and biological evo-

lution. In M. Barbieri (Ed.), Introduction to biosemiotics . Berlin: Springer.Battail, G. (2006b). Error-correcting codes and genetics. tripleC, 4 (2), 217229. http://triplec.uti.at/ .Battail, G. (2006c). Should genetics get an information-theoretic education? IEEE Engineering in

Medicine and Biology Magazine, 25 (1), 3445.Battail, G. (2007) Impact of information theory on the fundamentals of genetics. In G. Witzany

(Ed.), Biosemiotics in transdisciplinary contexts . Helsinki: Umweb.Battail, G. (2008). Genomic error-correcting codes in the living world. Biosemiotics .

doi :10.1007/s12304-008-9019-z.Benner, S. (2008). Biology from the bottom up. Nature, 452 (7188), 692694.Berrou, C., Glavieux, A., & Thitimajshima, P. (1993). Near Shannon limit error-correcting coding

and decoding: Turbo-codes. In Proc. ICC93 (pp. 10641070). Switzerland: Geneva.Berrou, C., & Glavieux, A. (1996). Near optimum error correcting coding and decoding: Turbocodes. IEEE Transactions on Communications, 44 , 12611271.

Chaitin, G. (2005). Metamaths! New York: Pantheon.Dawkins, R. (1976). The selsh gene . Oxford: Oxford University Press (New edition, 1989).Dawkins, R. (1986). The blind watchmaker . New York: Longman.Guizzo, E. (2004). Closing in on the perfect code. IEEE Spectrum, 41 (3) (INT), 2834.
http://triplec.uti.at/http://triplec.uti.at/http://10.0.3.239/s12304-008-9019-zhttp://10.0.3.239/s12304-008-9019-zhttp://10.0.3.239/s12304-008-9019-zhttp://10.0.3.239/s12304-008-9019-zhttp://triplec.uti.at/

7/28/2019 Full Text 27

18/18

320 G. Battail

Pullen, S. W. (2005). Intelligent design or evolution? Why the origin of life and the evolution of molecular knowledge imply design . Raleigh: Intelligent Design.

Regis, E. (2008). What is life? Investigating the nature of life in the age of synthetic biology . NewYork: Farrar, Straus and Giroux.

Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27 , 379457, 623656.

Yockey, H. P. (1992). Information theory and molecular biology . Cambridge: CambridgeUniversity Press.

Yockey, H. P. (2005). Information theory, evolution, and the origin of life . Cambridge: CambridgeUniversity Press.

Full Text 27

Documents

Transcript of Full Text 27