Communication Complexity of Randomness...
Transcript of Communication Complexity of Randomness...
![Page 1: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/1.jpg)
of 17June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1
Communication Complexity of Randomness Manipulation
Madhu SudanHarvard University
Joint works with Mitali Bafna (Harvard), Badih Ghazi (Google), and Noah Golowich (Harvard)
![Page 2: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/2.jpg)
of 17
Randomness Processing Industry
Dispersers, Extractors, Merges, Condensers, PRGs …
Interest not in the exact function. (E=RSA most boring.)
Rather in its manipulation of distributions … Distribution of 𝑋𝑋 vs. that of 𝐸𝐸(𝑋𝑋)
Long history … omitted. Key ingredients single processor “𝐸𝐸” unknown source 𝑋𝑋 ∈ 𝒳𝒳
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 2
𝐸𝐸𝑋𝑋 𝐸𝐸(𝑋𝑋)
![Page 3: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/3.jpg)
of 17
2-player setting:
Is 𝛿𝛿 = 0 possible? If not minimize 𝛿𝛿! Etc. Alice + Bob can use private randomness Zero communication version: “Non-Interactive
Simulation” (NIS).
Distributed Randomness Processing
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 3
Alice 𝑈𝑈
Bob𝑌𝑌 𝑉𝑉
≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds
𝑋𝑋𝑋𝑋1,𝑋𝑋2, …
𝑌𝑌1,𝑌𝑌2, …
𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄
Infinite sequence
![Page 4: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/4.jpg)
of 17
A classical example
Zero communication 𝑋𝑋,𝑌𝑌 ∼ Unif 0,0 , 0,1 , 1,0 𝑈𝑈,𝑉𝑉 ∼ 𝜌𝜌-correlated bits.
𝑈𝑈,𝑉𝑉 ∼ Unif 0,1 , Pr 𝑈𝑈 = 𝑉𝑉 = 1+𝜌𝜌2
[Witsenhausen ‘70s]: Can’t achieve 𝜌𝜌 = 1. Not even 𝜌𝜌 = .51
(Naïve strategy achieves 𝜌𝜌 = 14)
Can achieve 𝜌𝜌 = 13
by a general method.
Best 𝜌𝜌? Open!!
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 4
![Page 5: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/5.jpg)
of 17
Witsenhausen …
Intermediate Problem: Output (𝐴𝐴,𝐵𝐵) w. 𝐴𝐴,𝐵𝐵 ∼ 𝑁𝑁 0,1with maximum correlation.
Definition: Max-Corr 𝜌𝜌 𝑋𝑋,𝑌𝑌 ≜ max𝑓𝑓,𝑔𝑔
𝔼𝔼𝑋𝑋,𝑌𝑌 𝑓𝑓 𝑋𝑋 𝑔𝑔(𝑌𝑌)
Where 𝑓𝑓,𝑔𝑔:Ω → ℝ,𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 = 0𝔼𝔼𝑋𝑋 𝑓𝑓 𝑋𝑋 2 = 𝔼𝔼𝑌𝑌 𝑔𝑔 𝑌𝑌 2 = 1
Thm 1: For any 𝑈𝑈,𝑉𝑉 output, 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 2: For Gaussian output (𝐴𝐴,𝐵𝐵), can achieve
𝜌𝜌 𝐴𝐴,𝐵𝐵 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Thm 3: For binary 𝑈𝑈,𝑉𝑉 , can achieve
1 −2 ⋅ cos−1 𝜌𝜌 𝑋𝑋,𝑌𝑌
𝜋𝜋≤ 𝜌𝜌 𝑈𝑈,𝑉𝑉 ≤ 𝜌𝜌 𝑋𝑋,𝑌𝑌
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 5
For Bernoulli/GaussiansMax-Correlation = Correlation
![Page 6: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/6.jpg)
of 17
“Tensorization”
Amortized Problems: Given (1) structure 𝑆𝑆 (graph, distribution, game) (2) product operation 𝑆𝑆⊗𝑡𝑡, (3) amortized measure 𝑀𝑀, compute lim
𝑡𝑡→∞𝑀𝑀 𝑆𝑆⊗𝑡𝑡
Shannon capacity, Compression length, Channel capacity, parallel repetition value of 2-prover game, Direct sum complexity, NIS
“Tensorizing bound”: Typically: 𝑀𝑀 𝑆𝑆⊗𝑡𝑡 ≥ 𝑀𝑀(𝑆𝑆). Find: ℳ(⋅) s.t. 𝑀𝑀 𝑆𝑆 ≤ ℳ(𝑆𝑆) and ℳ 𝑆𝑆⊗𝑡𝑡 = ℳ(𝑆𝑆)
Max-Correlation “Tensorizes” If 𝑋𝑋,𝑌𝑌 ∼ 𝜇𝜇 & 𝑋𝑋𝑡𝑡,𝑌𝑌𝑡𝑡 ∼ 𝜇𝜇⊗𝑡𝑡 then 𝜌𝜌 𝑋𝑋𝑡𝑡 ,𝑌𝑌𝑡𝑡 = 𝜌𝜌(𝑋𝑋,𝑌𝑌) Proof idea: 𝜇𝜇 described by matrix 𝑃𝑃; 𝜌𝜌 related to its
singular values, 𝜇𝜇⊗𝑡𝑡 described by 𝑃𝑃⊗𝑡𝑡
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 6
![Page 7: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/7.jpg)
of 17
Problem Single-shot Amortized
Compressibility P P
Channel capacity NP P
Problem Single-shot Amortized
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Problem Single-shot Amortized
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Non-interactive-simulation (“correlation”)
NP [0,𝐶𝐶𝐴𝐴]
Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴
Problem Single-shot Amortized
Compressibility P P
Channel capacity NP P
Independent set/Shannon capacity
NP [0,∞]
Value of 2-prover game NP [NP,∞]
Non-interactive-simulation (“correlation”)
NP [0,𝐶𝐶𝐴𝐴]
Communication Complexity [NP,Exp?] 0,𝐶𝐶𝐴𝐴
? P [NP,∞]
Amortized Embarassment
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 7
Glossary: 0 ≤ P ≤ 𝑁𝑁𝑃𝑃 ≤ EXP ≤ Computable≤ CA Computably Approximable ≤ ∞
![Page 8: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/8.jpg)
of 17
Complexity of NIS
“Computably approximable” [Ghazi,Kamath,S.], [De,Mossel,Neeman]2,
[Ghazi,Kamath,Raghavendra] There exists a finite time algorithm determining if we can
get 𝜖𝜖-close to (𝑈𝑈,𝑉𝑉) from i.i.d. samples of (𝑋𝑋,𝑌𝑌)
Idea: “Invariance Priniciple” [Mossel] Either there is a method using few samples of (𝑋𝑋,𝑌𝑌) or
many samples 𝑋𝑋1,𝑌𝑌1 , … , 𝑋𝑋𝑛𝑛,𝑌𝑌𝑛𝑛 each with low influence on 𝑈𝑈,𝑉𝑉 .
If latter, can replace 𝑋𝑋,𝑌𝑌 by finite # Gaussians. Gives computable upper bound on #samples needed to
get close to optimal strategy.
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 8
![Page 9: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/9.jpg)
of 17
Communication?
Restrict to “Common Randomness Generation” (𝑘𝑘-CRG): Output (𝑈𝑈 = 𝑉𝑉), with 𝑈𝑈 ∼ Unif( 0,1 𝑘𝑘)
First considered ~70 [Ahlswede-Csizar]: “Characterization”: Can communicate 𝑅𝑅 ⋅ 𝑘𝑘 +𝑜𝑜 𝑘𝑘 bits, where 𝑅𝑅 = min
ΠICint(Π)ICext Π
Computable? Computably approximable? For one-way communication: 𝑅𝑅 = 1 − 𝜌𝜌 𝑋𝑋,𝑌𝑌 2
[Zhao-Chia] (see also [Guruswami-Radhakrishnan, Ghazi-Jayram, S.-Tyagi-Watanabe])
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 9
Alice 𝑈𝑈
Bob 𝑉𝑉
≤ 𝐶𝐶 bits, ≤ 𝑟𝑟 Rounds
𝑋𝑋1,𝑋𝑋2, …
𝑌𝑌1,𝑌𝑌2, …
𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃 Goal: 𝑈𝑈,𝑉𝑉 ≈𝛿𝛿 𝑄𝑄
![Page 10: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/10.jpg)
of 17
This talk: Rounds in CRG
Does increasing #rounds ease CRG? (For some (𝑋𝑋,𝑌𝑌)?) … Is the following true?
Question: ∀𝜖𝜖 > 0 , 𝑟𝑟, ∃(𝑋𝑋,𝑌𝑌) s.t. ∀𝑘𝑘 ∃ 𝑟𝑟, 𝜖𝜖𝑘𝑘 -protocol for 𝑘𝑘-CRG 𝑋𝑋,𝑌𝑌 No 𝑟𝑟 − 1, 1 − 𝜖𝜖 .𝑘𝑘 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)
Still don’t know. Partial progress. Thm. [BGGS’19] : ∀𝑛𝑛,𝑘𝑘, 𝑟𝑟 there exists 𝑋𝑋,𝑌𝑌 s.t.
∃ 𝑟𝑟,𝑂𝑂(𝑟𝑟 log 𝑛𝑛 -protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌).
No 𝑟𝑟2− 3, min 𝑘𝑘, 𝑛𝑛
polylog 𝑛𝑛-protocol for 𝑘𝑘-CRG(𝑋𝑋,𝑌𝑌)
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 10
![Page 11: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/11.jpg)
of 17
Our Source:
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 11
𝑖𝑖0
𝜋𝜋1
𝜋𝜋2
𝜋𝜋3
𝜋𝜋𝑟𝑟
⋮
𝐴𝐴1 𝐴𝐴2 𝐴𝐴3 𝐴𝐴𝑛𝑛 𝐵𝐵1 𝐵𝐵2 𝐵𝐵3 𝐵𝐵𝑛𝑛… …
Alice’s Inputs𝑋𝑋 = (𝜋𝜋1,𝜋𝜋3, … ;𝐴𝐴1, … ,𝐴𝐴𝑛𝑛)
Bob’s Inputs𝑌𝑌 = (𝑖𝑖0;𝜋𝜋2,𝜋𝜋4, … ;𝐵𝐵1, … ,𝐵𝐵𝑛𝑛)
𝑖𝑖0 ∼ [𝑛𝑛]
𝜋𝜋1, … ,𝜋𝜋𝑛𝑛 ∼ 𝑆𝑆𝑛𝑛𝐴𝐴1, … ,𝐴𝐴𝑛𝑛 ∼ 0,1 𝑘𝑘
𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ∼ 0,1 𝑘𝑘
𝐴𝐴𝑖𝑖𝑟𝑟 = 𝐵𝐵𝑖𝑖𝑟𝑟where 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1
“Pointer Chasing Source” (PCS)
![Page 12: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/12.jpg)
of 17
CRG from Pointer Chasing Source
Easy direction easy: (𝑟𝑟, 𝑟𝑟 log𝑛𝑛)-protocol for 𝑘𝑘-CRG Hardness?
Pointer chasing is hard: [Duris-Galil-Schnitger, Nisan-Wigderson,…].
CRG from PCS requires pointer chasing? No! Can solve problem without pointers! Small non-deterministic complexity! Hardness needs to use hardness of
disjointness? And of pointer-chasing? How to combine modularly?
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 12
![Page 13: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/13.jpg)
of 17
Our solution: Pointer Verification Problem
PVP: Distinguish 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉YES from 𝑋𝑋,𝑌𝑌 ∼ 𝑃𝑃𝑉𝑉No 𝑃𝑃𝑉𝑉YES:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟 𝑃𝑃𝑉𝑉NO:𝑋𝑋 = 𝜋𝜋1,𝜋𝜋3, …𝜋𝜋𝑟𝑟−1 ; 𝑌𝑌 = 𝑖𝑖0, 𝑖𝑖𝑟𝑟′ ; 𝜋𝜋2,𝜋𝜋4, …𝜋𝜋𝑟𝑟
YES instance satisfy: 𝑖𝑖𝑗𝑗 = 𝜋𝜋𝑗𝑗 𝑖𝑖𝑗𝑗−1 NO instance: 𝑖𝑖𝑟𝑟′ random
Claims: Hardness of (Unique) Set Disjointness ⇒
Protocol for 𝑘𝑘-CRG(PCS) solves PVP.
No 𝑟𝑟2− 𝑂𝑂 1 , 𝑛𝑛
polylog 𝑛𝑛-protocol for PVP.
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 13
![Page 14: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/14.jpg)
of 17
PVP ≤ CRG
Let 𝜇𝜇𝑋𝑋𝑌𝑌 denote dist. of PCS source. 𝜇𝜇𝑋𝑋, 𝜇𝜇𝑌𝑌 marginals
CRG Hard ⇐ 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 (to 𝑟𝑟2
,𝐶𝐶 -protocols)
Intermediate distribution: 𝜇𝜇mid 𝑋𝑋 = (𝜋𝜋odd,𝐴𝐴1, … ,𝐴𝐴𝑛𝑛); 𝑌𝑌 = 𝑖𝑖0,𝜋𝜋even,𝐵𝐵1, … ,𝐵𝐵𝑛𝑛 ,
with 𝐴𝐴𝑗𝑗 = 𝐵𝐵𝑗𝑗 for random 𝑗𝑗. (Correlation exists but pointers don’t point to it.)
Claims: 𝜇𝜇𝑋𝑋𝑌𝑌 indist. from 𝜇𝜇mid ⇐ PVP is hard 𝜇𝜇mid indist. from 𝜇𝜇𝑋𝑋 × 𝜇𝜇𝑌𝑌 ⇐ (Unique) disjointness
hard.
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 14
𝑖𝑖
⋮
![Page 15: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/15.jpg)
of 17
Bulk of paper: PVP is hard
Main idea: “Round elimination” a la NW ’93 No black-box use No simple variant
Induction on #rounds Many invariants … roughly
𝐻𝐻 𝜋𝜋odd,𝜋𝜋even transcript) ≈ 𝑀𝑀𝑀𝑀𝑀𝑀 − 𝑂𝑂 𝐶𝐶 𝐻𝐻 𝑖𝑖𝑡𝑡 transcript) ≈ log𝑛𝑛 − 𝑜𝑜 1 𝑋𝑋 ⫫ 𝑌𝑌 | path, transcript
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 15
![Page 16: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/16.jpg)
of 17
Conclusions
Many interesting problems in distributed randomness manipulation.
Complexity of the “Single-letter characterizations”.
Tight characterization of round complexity of CRG.
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 16
![Page 17: Communication Complexity of Randomness Manipulationmadhu.seas.harvard.edu/talks/2019/IITB-CRG.pdf · June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 1 of 17 Communication](https://reader035.fdocuments.us/reader035/viewer/2022070803/5f0333117e708231d408078b/html5/thumbnails/17.jpg)
of 17
Thank You!
June 03, 2019 IITB: Comm. Comp. Randomness Manipulation 17