1

16.548 Notes II

Jay Weitzen

University of Massachusetts Lowell

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Information and Communication

16

Key Slide

17

18

Hard decision vs Soft Decision

19

20

21

22

23

24

25

26

27

Applications of Information Theory: Compression

28

Shannon’s First Theorem: A.K.A Source Coding Theorem

29

Shannon’s source coding theorem

30

31

32

33

Rate of a Source Code

34

2nd order Block Codes and Huffman Encoding

35

36

37

38

39

40

Some Definitions

41

Higher Order Codes Converge

42

43

44

45