6 Information in the Mathematical Theory of Communication

download 6 Information in the Mathematical Theory of Communication

of 12

Transcript of 6 Information in the Mathematical Theory of Communication

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    1/12

    silan tharakan

    INFORMATION IN THEMATHEMATICAL

    THEORY OF

    COMMUNICATION

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    2/12

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    3/12

    silan tharakan

    Historical Development developed by Norbert Weiner

    According to Weiner, any organization is held

    together by the possession of means foracquisition, use, retention and transmissionof information

    information theory has developed primarily as

    a mathematical theory of communication

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    4/12

    silan tharakan

    Model of a Communication

    System purpose of a communication system is to

    reproduce at the destination a message

    selected at the source

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    5/12

    silan tharakan

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    6/12

    silan tharakan

    Each sound, inflection, pitch, etc., of thehuman voice is a message in the telephone

    system Considerably more messages are required to

    relay the voice than the coded letters andnumbers

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    7/12

    silan tharakan

    Mathematical Definition of

    Information It is the average number of binary digits which must

    be transmitted to identify a given message from the

    set of all possible messages to which it belongs

    The message to be transmitted is encoded, thecodes are sent over the channel, and the decoderidentifies the message intended by the codes

    size of the code is dependent on the coding scheme

    and the number of possible messages

    coding scheme for information theory is assumed to

    be binary

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    8/12

    silan tharakan

    communication system is required to respond toonly two responses, "yes" or "no,"

    then the system needs to transmit one of only twopossible signals1 and 0

    E.g. a system used to transmit birthday greetings

    senders may not write their own greetings-they mayonly select from several standard messages

    Assume there are eight such messages

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    9/12

    silan tharakan

    The information content(or code size in bits)

    may be generalized as:

    where n = the totalnumber of possiblemessages

    If n = 8, I = 3 bits

    2logI n

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    10/12

    silan tharakan

    Reduction of Uncertainty Information reduces uncertainty

    Partial information will reduce the uncertainty but noteliminate it

    in the case of the eight birthday messages, three

    bits are required to eliminate uncertainty completelyby identifying the exact message

    transmitting a single bit selects one of the twogroups and reduces the possibilities from eight tofour as shown in figure following

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    11/12

    silan tharakan

  • 8/6/2019 6 Information in the Mathematical Theory of Communication

    12/12

    silan tharakan

    Redundancy Redundancy reduces the efficiency of a particular

    transmission

    because more codes are transmitted than are strictly

    required to encode the message The transmission of redundant data allows the

    receiver to check whether the received message iscorrect and may allow the original message to bereconstructed

    redundancy means that the listener need not readand decode every letter in order to understand themessage