INFORMATION THEORY & CODING (ITC)

APPLICATIONS OF INFORMATION THEORY PREPARED BYDARSHAN BHATT(150320704001)11-12-2015ITC (2710502)1

1

INTRODUCTIONInformation theory was introduced by Shannon in the late 1940 as a mathematical theory.It is a branch of applied mathematics, electrical engineering & computer science.Information theory is concerned with the analysis of communication systems.

11-12-2015ITC (2710502)2

Applications of Information TheoryEfficiency and Capacity of the computers and similar devicesVideo abstraction extractionSecure Data TransmissionChannel CapacitySpeech CodingWaveform CodingLossless data compression Cryptography

11-12-2015ITC (2710502)3

PART 1

11-12-2015ITC (2710502)4Application of Information Theory to Analysis of Efficiency & Capacity of Computers & Similar devices

Aim : To find the capacity & efficiency of the computersA new approach to definition & estimation of the computers efficiency and capacity can be presented with the help of information theory principles.Definition is based on the analysis of processor instructions and accessible memory.This approach is based on Execution time of each instructions and the speed of each kind of memory.Application of Information theory : Analysis of efficiency & capacity of computers11-12-2015ITC (2710502)5

Computer capacity with certain assumptionsConsideration : There is a program and data in binary words - P and D stored in memory M of computer.

Task of computer : < P , D >< P , D > having a sequence of instructions X (P,D) = x1x2x3......xi

For two computer task < P1 , D1 > and < P2 , D2 > are different if the sequences are different.

Execution time of a sequence of instruction = (x)

11-12-2015ITC (2710502)6

Capacity of computerWhat is computer Capacity ?Let there be a computer with a set of instructions I and let (x) be the execution time of an instruction x I: The computer capacity C(I) is defined as follows:

Here, N(T) = Total numbers of computer tasks evaluated in time T

11-12-2015ITC (2710502)7

Some notation & definition from Information TheoryWhen a computer is used for solving problems of a certain kind. For example, one computer can be a Web server, another can be used for matrix multiplicationThe computer efficiency depends on the problems the computer has to solve.Let there be a process z = z1;z2;z3... generating letters from a finite alphabet A, The nth order Shannon entropy and the limit Shannon entropy is defined as follows:

11-12-2015ITC (2710502)8

Pz(u) = probability of generating letters from finite alphabet A 8

Efficiency of computers Suppose that there is a computer with a set of instructions I and let (x) be the execution time of an instruction x I: Let this computer be used for solving such a randomly generated sequence of computer tasks, that the corresponding sequence of the instructions z = z1, z2, z3,......zi I, Then the efficiency is defined as

Here c(I,z) is the quantity of information per time unit.

11-12-2015ITC (2710502)9

Here h(z) is Shannon entropy which is nothing but a quantity of information per letter. And denominator in above equation shows average execution time of an instructions. So c(I,z) is the...........9

Computer capacity from ShannonShannon showed that the channel capacity C(I) is equal to the logarithm of the largest real solution X0 of the following equation :

In other words C(I) = log X0

11-12-2015ITC (2710502)10

ConclusionLet there be a computer with a set of instructions I and let (x) be the execution time of x I.Then, The alphabet capacity C(I) = log X0 The efficiency is maximal if the sequences of instruction are generated by an i.i.d source with probability p(x) = (X0)-(x)

11-12-2015ITC (2710502)11

X0 = largesT real solution of previous slides equationI I D = Independent identical distributed 11

11-12-2015ITC (2710502)12PART 2

Application of Information Theory in Video Abstraction Extraction

Application of information theory in Video Abstraction With the development of internet and multi-media business the video becomes one of the main carrier that the modern information spread.

An urgent problems are how to reduce cost of video storage, how to improve efficiency, availability and accessibility of the video.

Video Abstraction is the solution of these problems.11-12-2015ITC (2710502)13

Video AbstractionThe concept of video abstraction comes from that of the text summary.

It is a brief summary for a long video.

It is to generate compact still or moving image sequence which can make full performance of the video content.

11-12-2015ITC (2710502)14

Video AbstractionVideo SkimmingVideo SummaryIt is a collection of key frames extracted from the original video.It makes comprehensive use of original video, audio, text.

11-12-2015ITC (2710502)15

So video summary is quick and simple to build and skimming is more complex to deal. Video summary give the general description of the original video.15

Information theory in Key frame ExtractionA video is combination or the group of a sequence of images to form a moving picture.The concepts of information theory is used in extraction the key frames or an important frames from the original long size videos.According to the amount of mutual information between two key frames, the initial extracted key frames were re-extracted to form the video summary.To measure the amount of information in key frame Information Entropy is used as a standard. 11-12-2015ITC (2710502)16

Concept of Mutual Information (MI) 11-12-2015ITC (2710502)17

Identification of similar framesFrom the concept of mutual information, we can find similar frame structures in the long duration video.If the value of MI between the two key frames is large, then these two key frames have a great similarity.The MI between first two key frames t and t+1 is :

11-12-2015ITC (2710502)18

ConclusionVideo abstraction is the brief summary of the original long duration video. To reduce the cost of video storage, and to improve the use efficiency, availability & accessibility of the video we have to use this concept of abstraction.Video abstraction is done by using two dimentional histogram and the important concept of information theory.11-12-2015ITC (2710502)19

19

ReferencesYin gian, Dong Meng, Yu xiu. Application of information theory in video abstraction, IEEE Transaction on Environmental science & information application technology, 2010.

Boris ryabko, Application of information theory to analysis of efficiency & capacity of computers & similar devices. IEEE Region 8, July 2010.

www.wikipedia.org

11-12-2015ITC (2710502)20

THANK YOU

2111-12-2015ITC (2710502)

21