Randomness as a Resource in Modern Communication and ... · Randomness as a Resource in Modern...

Randomness as a Resource inModern Communication and Information Systems

Holger BocheTechnical University MunichDepartment of Electrical and Computer EngineeringChair of Theoretical Information Technology – LTI

Joint Work with Christian Deppe, TUM, LNT

IEEE Statistical Signal Processing Workshop 201810-13 JuneFreiburg, Germany

Contents

• Motivation• The Classical Communication (Shannon Picture)• Local and Global Randomness as Ressources• Physical Layer Security• A New Communication Paradigm (Identification)• Physical Layer Security and Identification• Robust Channel Models• Conclusions

Holger Boche (TUM) 2

Chair of Theoretical Information Technology

Department of Electrical and Computer Engineering

Technical University Munich

Holger Boche (TUM) 3Source Cliparts: Openclipart.org In Cooperation with Christian Arendt, BMW Group, LTI TU München




The Classical Communication (Shannon Picture)

mAlice

Encoder Noisy Channel W nDecoder

Bobm

xn ∈X n yn ∈ Y n

• Alice has to transmit a message m ∈M = {1,2, . . . ,M} to Bob

• Alice uses a block code X n = {0,1, . . . ,q−1}n

• W = {W (y |x) : x ∈X ,y ∈ Y } is a stochastic matrix.

• The probability for a sequence y ∈ Y n to be received if xn ∈X n:

W n(yn|xn) =n

∏t=1

W (yt |xt)

• Bob receives a word in Y n.

Goal: Bob has to decode the correct message with a small decoding error=⇒ Finding the correct answer to: “What was Alice’s message?”






Practical Question: What is the largest rate (R = logMn ) of (almost) error free communication from Alice

to Bob?

=⇒ Message transmission capacity C

= max(I(P;W ))

=⇒ Size of message set |M|= 2Cn

I(X ∧Y ) := H(Y )−H(Y |X )

If P is a probability distribution on X and W = {W (y |x) : x ∈X ,y ∈ Y } a stochastical matrix, wedefine

I(P;W ) := I(X ∧Y )

where X is a RV with distribution P and Y has conditional distribution W (·|x) given X = x .

The Shannon Picture is the theoretical framework for all existing communication systems,storage systems, and information processing systems.






Practical Question: What is the largest rate (R = logMn ) of (almost) error free communication from Alice

to Bob?

=⇒ Message transmission capacity C= max(I(P;W ))

=⇒ Size of message set |M|= 2Cn

I(X ∧Y ) := H(Y )−H(Y |X )

If P is a probability distribution on X and W = {W (y |x) : x ∈X ,y ∈ Y } a stochastical matrix, wedefine

I(P;W ) := I(X ∧Y )

where X is a RV with distribution P and Y has conditional distribution W (·|x) given X = x .

The Shannon Picture is the theoretical framework for all existing communication systems,storage systems, and information processing systems.





Randomness as Ressource

Common Randomness γ ∈ Γ

γγ

⇓ ⇓

Alice’s local randomness Bob’s local randomness

Randomized encoding Randomized decoding

mAlice


Bobm


The upper part, that is called the Common Randomness part, will be needed later.





Deterministic vs. Random(local randomness at the transmitter)A (deterministic) (n,M,λ ) code for W is a set of pairs {(ui ,Di) : i ∈M }

ui ∈X n,Di ⊂ Y n for all i ∈M (1)

Di ∩Dj = /0 for all 1≤ i, j ≤ n, i 6= j (2)

W n (Di |ui)≥ 1−λ for all i ∈M (3)

A randomized (n,M,λ ) code for W is a set of pairs {(Q (·|i) ,Di) | for all i ∈M }

Q (·|i) ∈P (X n) , Di ⊂ Y n for all i ∈M (4)

Di ∩Dj = /0 for all 1≤ i, j ≤ n, i 6= j (5)

∑xn∈X n

Q (xn|i)W n (Di |xn)≥ 1−λ1 for all i ∈M (6)





Shannon’s Coding Theorem

Theorem (Channel Coding Theorem, Shannon 1948)

Let λ ∈ (0,1) be fixed. Then

limn→∞

logM(n,λ )

n= max

P∈P(X )I(P,W ) , C (7)

Lemma

Let W be a DMC, λ < 1/2. A deterministic (n,M,λ ) transmission code for W exists if and only if arandomized (n,M,λ ) transmission code exists.

No performance gain with local randomness. – But what can we achieve with common randomnessbetween Alice and Bob?





Shannon Picture with Common Randomness

Common Randomness γ ∈ Γ

γγ

mAlice


Bobm


Easy Consequence of Shannon’s Theorem: We achieve the same capacity, i.e. with commonrandomness we cannot achieve any performance increase.





Physical Layer Security• Classical approach to support security in communication systems⇒ separation of communication

and encryption⇒ in general no provable security

• New approach: embedded security with information theoretic approachGoal:− Alice has to transmit a message to the legitimate receiver Bob− Bob has to decode the correct message with small decoding error− Non legitimate receiver Eve is not able to decode the message

Alice Noisy Channel W Bobx ∈X y ∈ Y

Noisy Channel V Evez ∈Z

What is the largest rate of (almost) error free secure communication from Alice to Bob?=⇒ Secure message transmission capacity CS= max(I(U ∧Y )− I(U ∧Z ))

Message size |M|= 2CSn with provable security.Here a randomized encoding is necessary! The local randomness is absolutely important.










What is the largest rate of (almost) error free secure communication from Alice to Bob?=⇒ Secure message transmission capacity CS

= max(I(U ∧Y )− I(U ∧Z ))











What is the largest rate of (almost) error free secure communication from Alice to Bob?=⇒ Secure message transmission capacity CS= max(I(U ∧Y )− I(U ∧Z ))






Physical Layer Security

An (n,M,λ ) code for the wiretap channel is defined as a system{

(Qi ,Di) : 1≤ i ≤M}, where for all i

Qi ∈P(X n) and Di ⊂ Y n. It is required

1. for all i 6= j : Di ∩Dj = /0

2. for all i : ∑xn∈X n Qi(xn)W nt (Di |xn)≥ 1−λ ,

3. I(U ∧Z n)≤ λ , where U is a uniformly distributed random variable taking values in {1, . . . ,M} andZ n is the resulting random variable at the output of the channel V of the wiretapper.

The secrecy capacity of the general wiretap channel was determined by Csiszar and Körner. It is

Cs = maxU→X→YZ

I(U ∧Y )− I(U ∧Z ).

Message size: Mn = 2CSn





A New Communication Paradigm• New applications are event-driven, e.g. Industry 4.0, V2X, V2V, ... =⇒ Identification task

Goal:− Alice has to transmit a message m ∈N to Bob− Bob is interested in message m′, and he has to decide “m′ is transmitted or not ?”− Alice has no knowledge about m′

=⇒ Bob’s question: “Is the message m′?”

Alice m Noisy Channel W Bob m′ yes/nox ∈X y ∈ Y

Practical Question: What is the largest rate of (almost) error free identification?

=⇒ Identification capacity CID= C = max(I(X ∧Y ))

However: Message size |N|= 22Cn=⇒ double exponential increase

Here again randomized encoding is necessary! The local randomness is absolutely important.Otherwise |N|= 2Cn.










=⇒ Identification capacity CID

= C = max(I(X ∧Y ))












=⇒ Identification capacity CID= C = max(I(X ∧Y ))







A New Communication ParadigmA randomized (n,N,λ1,λ2) identification code is a set of pairs {(Qi ,Di) |i = 1, . . . ,N} with

Qi ∈P (X n) , Di ⊂ Y n for all i = 1, . . . ,N

and with errors of first resp. second kind bounded by

∑x∈X n

Qi (xn)W n (Di |xn)≥ 1−λ1 for all i = 1, . . . ,N (8)

and

∑x∈X n

Qj (xn)W n (Di |xn)≤ λ2 for all i, j = 1, . . . ,N, i 6= j (9)

The receiver who is interested in message i will decide that his message was transmitted iff thereceived channel output is in Di , otherwise he will deny that message i was sent.





A New Communication ParadigmLocal Randomness at Alice site

⇓

Alice’s local randomness

Randomized encoding

mAlice


Bobm transmitted or not






A New Communication Paradigm

Theorem (Ahlswede, Dueck, Han, Verdú)

Let N (n,λ ) be the max. number, such that an (n,N,λ1,λ2) ID code exists with λ1,λ2 ≤ λ . Then

limn→∞

log logN (n,λ )

n= C for all λ ∈ (0,1/2) (10)

where the constant C denotes the Shannon’s channel capacity.





A New Communication ParadigmDeterministic Encoding and Decoding

mAlice


Bobm transmitted or not


=⇒ Mn = 2Cn

=⇒ with local resource of randomness at the transmitter we have an exponential performanceincrease.

Deterministic Approach Local Randomization

Mn = 2Cn =⇒ Mn = 22Cn





Physical Layer Security and Identification• New approach: embedded security and identification

Goal:− Alice has to transmit a message to the legitimate receiver Bob− Bob is interested in message m′, and he has to decide “m′ is transmitted or not ?”− Alice has no knowledge about m′

− Non legitimate receiver Eve is not able to identify any message



What is the largest rate of (almost) error free secure identification from Alice to Bob?

=⇒ Secure message transmission capacity CSID=

{0 if CS > 0CID otherwise.

Message size |N|= 22Cnwith provable security if CS > 0.

Here a randomized encoding is necessary! The local randomness is absolutely important.











=⇒ Secure message transmission capacity CSID

=














=⇒ Secure message transmission capacity CSID=








Physical Layer Security and IdentificationA randomized (n,N,λ ) identification code of a wiretap-channel is a family of pairs{(Qi ,Di) |i ∈ {1, . . . ,N}} with Qi ∈P (X n) , Di ⊂ Y n, ∀ i ∈ {1, . . . ,N} such that∀ i, j ∈ {1, . . . ,N}, i 6= j

∑xn∈X n

Qi (xn)W n (Di |xn)≥ 1−λ , (11)

∑xn∈X n

Qj (xn)W n (Di |xn)≤ λ , (12)

∑xn∈X n

Qj(xn)V n(E |xn) + ∑xn∈X n

Qi(xn)V n(E c|xn)≥ 1−λ (13)

for any pair (i, j) with i 6= j and any E ⊂Z n.

Theorem (Ahlswede, Zhang, 1995)

Let C be the Shannon capacity of the channel W and let CS be the secrecy transmission capacity ofthe wiretap channel, then

CSID =

{0 if CS > 0C otherwise.

Message size |N|= 22Cn





A Robust Channel ModelSender Wt

State selector t ∈ θ

Receiver

• In this model the channel uncertainty is modeled by a given set of channels.• The communication participants know this set, but they do not know which channel of this set

describes the channel actual used.• This compound channel (CC) model was introduced by Blackwell, Breiman, and Thomasian.• It can be considered as a channel with state selector, choosing t , but the sender and the receiver

do not know his selection.

Let W the set of possible channels, then

C = maxP

minW∈W

I(P;W ). (14)





A Channel Model with a JammerSender Wsn

Jammer chooses sn ∈S n

Receiver

• This setting is modeled by a given set of channels.• The communication participants are aware of the state set but not of the actual realization

determining the probabilistic channel law.• So they do not know the jamming strategy of the jammer.• In every time step the state of the channel can be changed by the jammer.• The channel is called an Arbitrarily Varying Channel (AVC).• In the calculation, a new effect occurs: symmetrizability.• The intuitive meaning of this is that the jammer can choose the state of the channel such that any

two codewords, x and x ′, may be confused by the receiver. In this situation, the decoder will beunable to tell if the transmitted codeword was x or x ′. When a channel is symmetrizable, it is notpossible to transmit or identify a message.





A Channel Model with a Jammer

Definition

An AVC W is symmetrizable if there exists a channel U : X →P(S ), such that

∑s

U(s|x ′)W (y |x ,s) = ∑s

U(s|x)W (y |x ′,s) (15)

for all x ,x ′ ∈X and for all y ∈ Y .

Csiszár and Narayan provided the following expression.

Theorem (Csiszár, Narayan, 1988)

C =

{0 if W is symmetrizablemaxP minW∈W I(P;W ) otherwise,

where W = {∑s P(s)W (·|·,s) : P ∈P(S )}.





A Robust Channel Model with WiretapperSender Wt

Vt

State selector

t ∈ θ = {1, . . . ,T}

Receiver

Wiretapper

t

• We assume (to model a strong wiretapper) that the wiretapper knows the actual channel.• A code for the channel conveys information to the legal receiver such that the wiretapper knows

nothing about the transmitted information.• As before, a state selector chooses t , but the sender and the receiver do not know his selection.

Now the wiretapper gets t .• The CC belongs to the class of channels, where for transmission and identification there is no

strong converse.Holger Boche (TUM) 25




A Robust Channel Model with Wiretapper

Theorem (Bjelakovic, B., Sommerfeld, 2013)

The secrecy capacity is given by

CS(W ,V ) = limn→∞

1n

maxU→Xn→Y n

t Z nt

(mint∈θ

I(U ∧Y nt )−max

t∈θ

I(U ∧Z nt )),

where Yt is the resulting random variable at the output of the legal receiver channel and Zt is theresulting random variable at the output of the wiretap channel, if the channel is (Wt ,Vt). The maximumis taken over all random variables that satisfy the Markov chain relationships U→ X n

t → Y nt Z n

t .





A Channel Model with a Jammer and a Wiretapper

Common Randomness

Sender

local randomness

Wsn

Vsn


Eavesdropper

Receiver

• The additional aspect of security is modeled by a wiretap channel.

• There is no single-letter formula for the capacity.






Theorem (Nötzel, Wiese, B., 2016)

For the AVWC (W ,V ), we have

CS,ran(W ,V ) = limn→∞

1n

sup{U,Xn,Y n

q ,Z nsn}

(min

q∈P(S )I(U ∧ Y n

q )− maxsn∈S n

I(U ∧ Z nsn)),

where the supremum is over the set of families of RVs

{U, X n, Y nq , Z

nsn : q ∈P(S ),sn ∈S n}. (16)

U assumes values in some finite subset of the integers. The values of X n lie in A n, those of Y nq in Bn,

those of Z nsn in C n, and such that for every q ∈P(S ) and sn ∈S n,

PUXnY nq Z n

sn(u,xn,yn,zn) = PU(u)PXn|U(xn|u) (17)(

n

∏i=1

[∑

s∈Sq(s)Ws(yi |xi)

])V n(zn|xn,sn).

PU and PXn|U may be arbitrary probability distributions and stochastic matrices, respectively.






Using the capacity for random codes and Ahlswede’s elimination technique the following result wasobtained.

Theorem (Bjelakovic, B., Sommerfeld, 2012)

CS(W ,V ) =

0 if CS(W ,V ) = 0 orW symmetrizable

CS,ran(W ,V ) otherwise.





Robust and Secure IdentificationSender Wt

Vt

State selector

t ∈ θ = {1, . . . ,T}

Receiver

Wiretapper

t

Same model like before for robust transmission, new goal: Identification.

Theorem (Dichotomy Theorem, B., Deppe, 2018)

Let C(W ) be the capacity of the CC W and let CS(W ,V ) be the secrecy capacity of the CWC(W ,V ), then

CSID(W ,V ) = C(W ), if CS(W ,V ) > 0,

andCSID(W ,V ) = 0, if CS(W ,V ) = 0.Holger Boche (TUM) 30




Identification with a Jammer and a WiretapperSender Wsn

Vsn


Eavesdropper

Receiver

Same model like before for robust transmission, new goal: Identification.

Theorem (B., Deppe, 2018)

Let Cran(W ) be the random coding capacity of the AVC W and let CSID(W ,V ) be the random codingsecrecy capacity of the AVWC (W ,V ). Then,

1. CSID(W ,V ) = Cran(W ), if CS,ran(W ,V ) > 0 and W is not symmetrizable.2. CSID(W ,V ) = 0, otherwise.





Identification with a Jammer and a Wiretapper

Remarks

1. The formula for secure identification with a jammer is a single-letter formula. The condition forpositivity is unfortunately a multi-letter formula.

2. CSID is generally a discontinuous function depending on (W ,V ).

3. We investigated the discontinuity behavior and the superactivation in detail.

4. For future applications with secure communication, it is important that the implemented codingmethods are efficient and provable. This proof is provided by the certification and standardization.This proof must be effective in the future, i.e. by an algorithm. In this case, it is necessary tounderstand exactly the analyzed properties of the corresponding capacity as a function ofchannels.

5. Here we have presented the theory of classical channels. For quantum channels, the theory wasdeveloped by Boche, Deppe, and Winter.





Conclusions: A New Communication Paradigm

Transmission - Shannon Picture

Sender chooses and sendsm ∈M = {1,2, . . . ,M = 2Cn}

channel

Receiver’s goal:What is the message?

Shannon 1948

Identification - New Communication Task

Sender chooses and sendsm ∈N = {1,2, . . . ,N = 22nC}local randomness

channel

Receiver’s goal:Is the message m′ ∈N ?

Ahlswede/Dueck 1989

First Robust Optimal Protocol; H.B., C. Deppe;

ISIT 2017, IEEETIFS 2018

⇒ Extension to Quantum Systems; H.B., C. Deppe, A. Winter

⇒ exponential performance increase





Conclusions: Security - New QualityTransmission - Shannon Picture

Sender chooses and sendsm ∈M = {1,2, . . . ,M = 2CSn}CS = max I(U ∧Y )− I(U ∧Z )local randomness

channel

Receiver’s goal:What is the message?

Wiretapper’s goal:What is themessage?

Wyner 1978

Identification - New Communication Task

Sender chooses and sendsm ∈N = {1,2, . . . ,N = 22nC}C = max I(X ∧Y ), if CS > 0local randomness

channelWiretapper’s goal:Is the messagem′′ ∈N ?

Receiver’s goal:Is the message m′ ∈N ?

First Secure Robust Optimal Protocol; H.B., C. Deppe; ISIT 2017

⇒ Extension to Quantum Systems; H.B., C. Deppe, A. WinterISIT 2018

⇒ exponential performance increase and we pay no price for Security





Conclusions: Extensions for New Approaches• Other communication/signal processing tasks for communication systems

• Secure data storage on public data base with double exponential growth⇒ “Big Data” is not a problem, ...

• Quantum Identification ... even new identification tasks according Quantum Physics

• Communication tasks for Quantum Computing• Quantum Information− Entanglement-Generation / Distributions− Strong Subspace− Private Keys ...

• Authentication via PUFs (Physical Unclonable Functions)

• Computations over channels





Randomness as a Resource in Modern Communication and ... · Randomness as a Resource in Modern...

Documents

Transcript of Randomness as a Resource in Modern Communication and ... · Randomness as a Resource in Modern...