⇒ tent KEIY "

mtgqfnettainisfix.2

Aslong as R > HWTEZ is achievable

→ You can turn this into O . error scheme by putting an indicator bit : If typical indicator bit is 1

and we do AEP encoding .If indicator is O then we encode the whole sequence xn with nlgl XI

and Avenge kyth : MAID (n( HKH )) + ( I - Pla's"D(nlykl)What about a converse result ?

Given n,

enc,

kc.

st.

MXNFITK for E arbitrarily small.

xtm . inDato Pnc

. ing .

nR3 Htm ) } Ilxn;m)3hIkn;xY=

HIM - Htxnkn) - NHHI - HKYIYZNHKI - HH - enlgtxl

⇒ Rs,

HW - elylxl - 1¥ as e→o 127K¥

Recall : 29/2016xn Thursday

fact" '

Contains all the probabilitySize zntkx

)

Let BY be the smallest set P[ B" ) > 1- s

.

Let eto la 's "l÷l Ain 't an ÷b" if ftp.ntlgaynt-o

Shannon Cipher with Stochastic Source and Small Error .

KEENKX aX

"

iid~Px WEENY X

f.

g

Assumptions: X "1K,

and k~dni.fm

Design : Block length n

RandomizedEncode : f :X "xR→W X

"- ( K ,w ) - In

Decoder

'

:g:Wxk→Xn}

fanned:{es

Goal : Reliability : P[ xntxn ] small.

secrecy :WIX " ( Perfect secrecy assumption )

#y, ( xp , )

Achievable rate :1 R

,Re ) is achievable if He >0 Fn ,f,g such that It HIKE

and WIX"

.

theorem The closure of the set of achievable rates ( R,

Ru ) is

{ ( R,

Rel :

RZHK)123Hk )k

Pro¥ (Achievability

:)First compress to rate HLX) +8

,then use one - time pad .

Xn MCompressor one - time

Rote Hk )Pod

:encoder

( Converse :) Assume (R, Re) is achievable.

For arbitrary E >o,

Fn ,fg givesobjective

: 3 HIW) ( uniform dist .

bud)3 HIWIK )3 IKYWIK )

=I( Xn ; W,

K )dttlxnik)=o

3I( xn ; In )

= HAY - Hulin )

= NHW - Htxnlx " ) ( X"

: :D )IHHHINKENIGKHHK)

⇒ 123 HHI . Elgklt Hk )

Let e→oSCEHO as E→o

RZHCX)

nrht H( K )

3M¥Ixnikwly°× "±W

!f9eYng=I( xn ; K

,w) - IfXnjw ) assumption

:

:Rp,

Hk )

Note Instead of WIM,

we could ask for a loosersecrecy

criterion

i.e. ntI( xnjw) ( {.

The some proof still holds.

÷ka.

"

Information leakage rate"

Channel CapacityAM

,FM radio

, Digital

Deforest :

Hamming(4¥a

kgtadebits

Pz 0

toB

,

bi,

bz,

bs,

be Pi,

Be,

B

BPHby I 0 I I I ° ° I

you send

R 04

I I I I O O 1

you get

"

ho.¥iY✓

b. 4 → Provided that there

/ ko is no more thansingle

bit flip

flip ×Hamming

code can always correct itself.

thisone !

Repetition code :

Repetition 11,3) :111

000

Bits are flipped with probability p .

Pri¥¥

?... .÷•...;•÷i/¥Y±%÷hB¥%↳

"

% .↳

10=0.01

• i

Hammigliitt 21,040gal 471 1 RodeWook:B

:b Replh ] ) sphtp'b

(what shannon showed)''

hptolp )

Unaded p1

GeneralSetting for Channel

Coding

MELIYp

xnNoissokmd

yn aPYIX9

TDiscrete

MemogkssChannel

We assume finitealphabets 1×1,181

Vi, Rhlxiyitlyilxig

" ) = Pypxilyilxil

( Note thatwe are notsaying

Pyyxn =P Pxx formemory

less channel ingenenl)

For a channel what feedback :Memaylessness *Pyyxn t.IMRyx

,

Let M be uniformly distributed

Defn : R is achievable if the >o,

Fn, fy at rate R such that P[ Mai ]< e

Defn ( Capacity ) : Let the capacity C be the suprmum of achievable rates R.

Theorem :

Company IN ;D ← with respect toPxR¥

problem statementgives thisExample : Binary noiseless channel

×o ;.IT ' I'±m§"x"k*, . ± bit

K -

aynoiseless channel

÷a• - . , Copnxax Ilxitknyxxtkxtlgk bitsa< +

FoKy

Non -

overlapping Noise

ו€. c=m¥ Hkklgkl⇐

Example 2 :

Noisy Typewriter :

1- bit

←-A ⇐

mp,qxIlxMl=m¥xlHHl-

HTHXD

B-a¥•c=yqxHH⇒

-

1%26-1=1%13bits

z

a-z

Example:

Jinay Symmetric channel

qq.tt#j,

cemapxxd" ' ' " " " "•=

=mpa×xHM- MB

Capacity

I = 1 - Hcp)

¥p

Example 4 : Binary Erasure Channel

X Y

o•#° c=m§x( HHI - HI

'msee HIP)1 •#

'

= mgxlnlx) - HKMI )⇐

H(X1Y=e)=fP)m%xHWtlpxl

÷-