idea1-2OutlinedΒ Β· 2019-08-08Β Β· Title: idea1-2Outlined Created Date: 7/23/2019 9:51:59 AM
Approximate Homomorphic Encryption and Privacy Preserving ...Β Β· β’ Idea1: Restriction of domain...
Transcript of Approximate Homomorphic Encryption and Privacy Preserving ...Β Β· β’ Idea1: Restriction of domain...
Jung Hee Cheon (SNU, CryptoLab)
Approximate Homomorphic Encryption and Privacy Preserving Machine Learning
Thanks to YongSoo Song, Kiwoo Lee, Andrey KIM
1. Homomorphic Encryption2. HEAAN3. Bootstrapping of HEAAN4. Toolkit for Homomorphic Computation
Outline
Integer-based HE scheme
- RAD PH[1]
β’ (Secret Key, Operation Key) = (a large prime π, π = ππ0)β’ Encryption: πΈππ π = π + ππ πππ πβ’ Decryption: πΈππ π πππ π = πβ’ πΈππ π1 + πΈππ π2 = π1 + ππ1 + π2 + ππ2
= π1 +π2 + π π1 + π2= πΈππ π1 +π2
- DGHV HE scheme (on β€2)[2]
β’ πΈππ(π β π2100) = π + 2100π + ππβ’ SECURE against quantum computingβ’ Use a polynomial ring π π = β€π[π₯]/(π₯π + 1)
Homomorphic Encryption
But, INSECURE!
2100π1 π1
2100π2 π2
2200π1π2 + 2100 π1π2 +π2π1 + π1π2
X
- On Polynomials (RingLWE)β’ [Gen09] ideal latticeβ’ NTRU: LTV12β’ Ring-LWE: BV11b, GHS13, BLLN13, HEAAN etc
- On Integers (AGCD)β’ [DGHV10] FHE over the Integers. Eurocrypt 2010β’ CMNT11, CNT12, CCKLLTY13, CLT14, etc
- On Matrices (LWE)β’ [BV11a] Efficient FHE from (Standard) LWE. FOCS11β’ Bra12, BGV12, GSW13
Fully Homomorphic Encryption
1. 2009~2012: Plausibility and Scalable for Large Circuitsβ’ [GH11] A single bit bootstrapping takes 30 minutesβ’ [GHS12b] 120 blocks of AES-128 (30K gates) in 36 hours
2. 2012~2015: Depth-Linear Constructionβ’ [BGV12] Modulus/Key Switchingβ’ [Bra12] Scale Invariant Schemeβ’ [HS14] IBM's open-source library Helib: AES evaluation in 4 minutes
3. 2015~Today: Usabilityβ’ Various schemes with different advantages (HEAAN, TFHE)β’ Real-world tasks: Big data analysis, Machine learningβ’ Competitions for Private Genome Computation (iDash, 2014~)β’ HE Standardization meetings (2017~)
Continued on next page
Summary of Progress in HE
Standardization: HomomorphicEncryption.org
1st Workshop (2017.7.13-14) 2nd Workshop (2018.3.15-16) 3rd Workshop (2018.10.20)
Jul 2017 in Microsoft, Redmond
Mar 2018 in MIT
Oct 2018 in Toronto
Best Performing HE Schemes
Type Classical HE Fast Bootstrapping Approximate Computation
Scheme[BGV12] BGV
[Bra12, FV12] B/FV[DM15] FHEW[CGGI16] TFHE
[CKKS17] HEAAN
PlaintextFinite Field
PackingBinary string
Real/Complex numbers
Packing
Operation Addition, Multiplication Look-up table & bootstrapping Fixed-point Arithmetic
LibraryHElib (IBM)
SEAL (Microsoft Research)
Palisade (Duality inc.)
TFHE(Inpher, Gemalto, etc.)
HEAAN (SNU)
Homomorphic Encryption
Exact Multiplication
1.23
4.56
0.78
0.91
5.6088
0.7098
3.98112624
2.34
5.67
8.91
0.23
13.2678
2.0493
27.18970254
108.2456382397886496
- The plaintext size is doubled after a multiplication.2 4
8
16
Approximate Multiplication
HEAAN [CKKS17]β’ Rescale after a multiplicationβ’ Tracing # of significandsβ’ Most data is processed approximately in Data analysis or ML
1.23
4.56
0.78
0.91
5.6188
0.7198
3.98112624
2.34
5.67
8.91
0.23
13.2778
2.0593
27.19970254
108.2556382397886496
2 2
2
2
HEAAN = ζ §ηΌ = Insightful Minds
[CKKS, AC17] Homomorphic Encryption for Arithmetic of Approximate Numbershttps://eprint.iacr.org/2016/421.pdf
Numerical Representation
- Encode π into an integer π β ππ₯ for a scaling factor π : 2 β¦ 1412 β 2 β 103
- FixedβPoint Multiplicationβ’ Compute π = π1π2 and extract its significant digits πβ² β πβ1 β π
: 1.234 Γ 5.678 = 1234 β 10β3 Γ 5678 β 10β3 = 7006652 β 10β6 β¦ 7007 β 10β3 = 7.007
Previous HE on LWE problem (Regev, 2005)
β’ ct = Encsk π , ct, sk =π
π‘π + π mod π
β’ Modulo π‘ plaintext vs Rounding operation
Approximate Computation
π
π/π‘
π
π‘
A New Message Encodingβ’ ct = Encsk π , ct, sk = ππ + π mod π
β’ Consider π as part of approximation error
Homomorphic Operations
Support for the (approximate) fixed-point arithmetic
- Leveled HE : π = ππΏ
HEAAN
Input π1 β ππ1, π2 β ππ2
Addition π1 + π2 β π β (π1 +π2)
Multiplication π = π1π2 β π2 β π1π2
Rounding πβ² β πβ1 β π β π β π1π2
π1 = ππ1 + π1
ππ2 = ππ2 + π2
π = π2π1π2 + π
πβ² = π β π1π2 + πβ²
π/π
Construction over the ringβ’ A single ctx can encrypt a vector of plaintext values π§ = (π§1, π§2, β¦ , π§β)β’ Parallel computation in a SIMD manner π§ βπ€ = (π§1π€1, π§2π€2, β¦ , π§βπ€β)
HEAAN Packed Ciphertext
Continued on next page
Let π = β€ π ππ + 1 and π π = π mod π = β€π π ππ + 1
β’ A ciphertext can encrypt a polynomial π π β π β’ Note (m0+m1X+ β¦ ) (m0β+m1βX+ β¦ )= m0m0β+(m0m1β +m0βm1)X+β¦
β’ Decoding/Encoding functionπ = β€ π ππ + 1 β β π ππ + 1 β βπ/2
where ππ + 1 = π β π1 π β π1β1 π β π2 π β π2
β1 β― π β π π 2 π β π π 2β1
β’ Example: π = 4, π1 = ππ₯π(ππ/4), π2 = ππ₯π(5ππ/4)
β’ π§ = 1 β 2π, 3 + 4π β¦ π π = 2 β 2 2 π + π2 β 2 π3
β’ β¦ π π = 2000 β 2828π + 1000π2 β 1414 π3
β’ π π1 β 1000.15 β 1999.55 π, π π2 β 2999.85 + 3999.55 π
RLWE-based HEAAN
π π β¦ π§ = π§1, β¦ , π§πβ2 , π§π = π ππ ,
Bootstrapping
DecryptionπΈπππ(πΎ)
πΈππ(π)
c = πΈπππ(π; π)
πΎ
π
πΈπππ π; πβ² , πβ²: small
Safe Box
[1] Cheon-Han-Kim-Kim-Song: Bootstrapping for Approximate Homomorphic Encryption. EUROCRYPT 2018
β’ Old Ciphertext with large noise
β’ Encrypted Secret Key
β’ New ciphertext with small noise
β’ Evaluate Decrypt circuit
Input
Output
Process
β’ Ciphertexts of a leveled HE have a limited lifespanβ’ Refresh a ciphertext ct = Encsk π
by evaluating the decryption circuit homomorphically: Decsk ct = π βΊ πΉct sk = π where πΉct β = Decβ ct
β’ Bootstrapping key BK = Encsk(sk): πΉct BK = πΉct Encsk sk = Encsk πΉct sk = Encsk(π)
β’ Homomorphic operations introduce errors Fine: πΉct BK = πΉct Encsk sk = Encsk πΉct sk + π = Encsk(π + π)
β’ How to evaluate the decryption circuit (efficiently)?: Decsk ct = ct, sk (mod π)
Bootstrapping
π·πππ π ππ‘ β¦ π‘ = ππ‘, π π β¦ π‘ π = π,
π‘ = ππΌ + π for some πΌ < πΎ
β’ NaΓ―ve solution: polynomial interpolation on [βπΎπ, πΎπ]β’ Huge depth, complexity & inaccurate result
Approximate Decryption
π·πππ π ππ‘ β¦ π‘ = ππ‘, π π β¦ π‘ π = π,
π‘ = ππΌ + π for some πΌ < πΎ
β’ Idea1: Restriction of domain π βͺ π
Approximate Decryption
β’ Idea1: Restriction of domain π βͺ π
β’ Idea 2: Sine approximation π βπ
2πsin π for π =
2π
ππ‘ (period:π , slope at 0=1)
π
1
π·πππ π ππ‘ β¦ π‘ = ππ‘, π π β¦ π‘ π = π,
π‘ = ππΌ + π for some πΌ < πΎ
Approximate Decryption
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
0
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
1
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
2
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
3
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
4
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
5
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
6
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formula
β’ πΆπ+1 π = πΆπ2 π β ππ
2 π , ππ+1 π = 2ππ π β πΆπ π
7
Bootstrapping of HEAAN
Sine Evaluation
- Direct Taylor approximationβ’ Huge depth & complexity
- Idea 1: Low-degree approx. near 0
β’ πΆ0 π = π=0π β1 π
2π ! π 2π 2π β cos π 2π
β’ π0 π = π=0π β1 π
2π+1 ! π 2π 2π+1 β sin( π 2π)
- Idea 2: Iterate by double-angle formulaβ’ πΆπ+1 π = πΆπ
2 π β ππ2 π , ππ+1 π = 2ππ π β πΆπ π
β’ Numerically stable & Linear complexityππ π β sin π
8
Bootstrapping of HEAAN
Iteration vs Direct Computation
- ππ(π) is obtained from π0(π) and πΆ0(π) by π iterationsβ’ One computation of Doubleβangle formula: 2 squarings + 1 additionβ’ π iterations take 2π squarings + π additionsβ’ Degree of ππ(π) β 2π
- Direct Taylor Approximationβ’ 2π multiplications to get 2π degree approximation π2π of sine function
Bootstrapping of HEAAN
Slot-Coefficient Switching
- Ringβbased HEAANβ’ Homomorphic operations on plaintext slots, not on coefficientsβ’ We need to perform the modulo reduction on coefficients
- Pre/post computation before/after sine evaluationβ’ Depth consumption: Sine evaluationβ’ Complexity: Slot-Coefficient switchings (# of slots)
- Performance of Bootstrapping
- Experimental Resultsβ’ 127 + 12 = 139 s / 128 slotsΓ12 bitsβ’ 456 + 68 = 524 s / 128 slotsΓ24 bits
Coefficient Plaintext Slots
β π ππ + 1 β π 2
π‘ π = π β πΌ π + π(π)
(π‘0, π‘1, β¦ π‘πβ1)
(π0, π1, β¦ , ππβ1)
π π = π0 +β―+ ππβ1ππβ1
β coeff to slotβ‘ sine eval
β’ slot to coeff
Bootstrapping of HEAAN
Speed of FHE
2011 2013 2014 2015 2016 2018
1-bit Amortized
1bit, 1800s
1bit, 0.7s
1bit, 0.052s
*120s, 250K bit
172s, 531bit 320s, 16K bit
1800s β 0.05s (1bit) 1800s β 0.00046s (Amortized)
: 30K~300K times
*[CHH18]Faster Homomorphic Discrete Fourier Transforms and Improved FHE Bootstrapping, eprint, 1073, 2018/ Intel Xeon CPU E5-2620 2.10GHz, 64RAM
[GH11] Implementing Gentryβs Fully-Homomorphic Encryption Scheme, Eurocrypt 2011.[CCK+13] Batch Fully Homomorphic Encryption over the Integers, Eurocrypt 2013.[CLT14] Scale-Invariant Fully Homomorphic Encryption over the Integers, PKC 2014.[HS15] Bootstrapping for Helib, Eurocrypt 2015[DM15] FHEW: Boostrapping Homomorpic Encryption in Less Than a Second, Eurocrypt 2015.[CGGI16] Faster Fully Homomorphic Encryption: Bootstrapping in less than 0.1 Seconds, Asiacrypt 2016.
Slide Courtesy of Xiaoqian Jiang (β³ = too small iteration β hard to adapt for other data)
Rank
1
3
β³
β³2
X
X
2017 Track 3: Logistic Regression Training on Encrypted Data
2018 Track 2 : Secure Parallel Genome Wide Association Studies using HE
Team Submission Schemes
End to End Performance Evaluation result ( F1- Score ) at different cutoffs
Running
time
(mins)
Peak
Memory (M)
0.01 0.001 0.0001 0.00001
Gold Semi Gold Semi Gold Semi Gold Semi
A*FHE
A*FHE -1 +
HEAAN
922.48 3,777 0.977 0.999 0.986 0.999 0.985 0.999 0.966 0.998
A*FHE -2 1,632.97 4,093 0.882 0.905 0.863 0.877 0.827 0.843 0.792 0.826
ChimeraVersion 1 + TFHE & HEAAN
(Chimera)
201.73 10,375 0.979 0.993 0.987 0.991 0.988 0.989 0.982 0.974
Version 2 215.95 15,166 0.339 0.35 0.305 0.309 0.271 0.276 0.239 0.253
Delft BlueDelft Blue
HEAAN1,844.82 10,814 0.965 0.969 0.956 0.944 0.951 0.935 0.884 0.849
UC San
Diego
Logistic Regr + HEAAN 1.66 14,901 0.983 0.993 0.993 0.987 0.991 0.989 0.995 0.967
Linear Regr 0.42 3,387 0.982 0.989 0.980 0.971 0.982 0.968 0.925 0.89
Duality IncLogistic Regr +
CKKS (Aka HEAAN),
pkg: PALISADE
3.8 10,230 0.982 0.993 0.991 0.993 0.993 0.991 0.990 0.973
Chi2 test 0.09 1,512 0.968 0.983 0.981 0.985 0.980 0.985 0.939 0.962
Seoul
National
University
SNU-1
HEAAN
52.49 15,204 0.975 0.984 0.976 0.973 0.975 0.969 0.932 0.905
SNU-2 52.37 15,177 0.976 0.988 0.979 0.975 0.974 0.969 0.939 0.909
IBMIBM-Complex CKKS
(Aka HEAAN),
pkg: HElIb
23.35 8,651 0.913 0.911 0.169 0.188 0.067 0.077 0.053 0.06
IBM- Real 52.65 15,613 0.542 0.526 0.279 0.28 0.241 0.255 0.218 0.229
Slide Courtesy of Xiaoqian Jiang
Packing Method
- HEAAN supports vector operations- How can we compute matrix operations for ciphertexts?
- Matrix Encoding method
How to Pack
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
Encode
1 2 β― 16π = πΈππ( )
Packing Method
- Matrix addition : trivial- Matrix multiplication : non-trivial (exercise)- Row/column rotation?
How to rotate
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
4 1 2 3
8 5 6 7
12 9 10 11
16 13 14 15
πππ‘ π, 1 = πΈππ( 16 1 β― 15 ) =
16 1 2 3
4 5 6 7
18 9 10 11
12 13 14 15
(wrong)
Packing Method
- Solution : using masking vector
How to rotate
16 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
β
0 1 1 1
0 1 1 1
0 1 1 1
0 1 1 1
=
0 1 2 3
0 5 6 7
0 9 10 11
0 13 14 15
πππ‘ π, 1 βπππ π =
4 5 6 7
8 9 10 11
12 13 14 15
16 1 2 3
β
1 0 0 0
1 0 0 0
1 0 0 0
1 0 0 0
=
4 0 0 0
8 0 0 0
12 0 0 0
16 0 0 0
πππ‘ π, β3 βπππ πβ² =
β΄ πΆπππ ππ‘ (π) = πππ‘ π, 1 βπππ π + πππ‘ π, β3 βπππ πβ² =
4 1 2 3
8 5 6 7
12 9 10 11
16 13 14 15
Message Space
- Bit-wise HE: β€2={0,1} with logical gatesβ’ Good at gate operations but slow at arithmetic.
- Word-wise HE: β or β€π with add. & mult.
β’ Good at poly evaluation but hard to evaluate non-poly function.
Idea: Use Polynomial Approximation for Non-Poly Functions!- Donβt use naΓ―ve Taylor Approx. It is local i.e. approx at a point- Minimize errors on an interval- Methods: Least Square, Chevyshev, Minimax β¦
Polynomial Approximation
Application : Homomorphic Logistic Regression [HHCP]- Need to evaluate sigmoid function : 1/(1+exp(-x))
Real Financial Large Data (Provided by KCB) About 400,000 samples with 200 features. Target value = βcredit scoreβ
Polynomial Approximation
Idea: πΆπππ π, π = limπββ
ππ
ππ+ππfor a,b>0
1. Compute approximately (Take only the msb of the results)
2. Choose k as a power of 2
3. Use iteration algorithms for division
Comparison
Method Scheme(Amortized)Running time
# of pairs Error
Bit-wiseHElib β 1 ms 1800
0TFHE β 1 ms -
Ours (Word-wise)
HEAAN 0.73 ms 216 < 2-8
General Max
πππ₯ π1, β¦ , ππ‘ = limπββ
π1π+1+β―+π
π‘π+1
π1π+β―+π
π‘π for ai>0
2nd πππ₯ π1, β¦ , ππ‘ = limπββ
π1π+1+β―+π
π‘π+1βπππ₯π+1
π1π+β―+π
π‘πβπππ₯π
Applications: k-max, threshold counting, clustering, β¦
Min/Max
Fast\Merge Sort: O(klogk)
- Comparison-based algorithm doesnβt work on HE- We cannot check min-max condition
Sorting Network: O(klog2k)
- Comparison network that always sort their inputs- Data-independent algorithm
Results
- 64 slots : about 12 min. (previous work : 42 min. [EGN+15])- 32,768 slots : about 10.5 hour (previous work: impractical)
Sorting on Encrypted Data
Basic Tools:
Packing, Matrix Operation, Comparison, Approximate inv/sigmoid,
Decision Tree: Packing, Comparison
Boosting: Comparison and Gradient Decent
Deep Neural Network: Fast matrix operations + Approximate
Convolution Neural Network: + Comparison
Toward Homomorphic Machine Learning
HEAAN natively supports for the (approximate) fixed point arithmetic
Ciphertext modulus log π = πΏ log π grows linearly
Useful when evaluating analytic functions approximately:β’ Polynomialβ’ Multiplicative Inverseβ’ Trigonometric Functionsβ’ Exponential Function (Logistic Function, Sigmoid Function)
Packing technique based on DFTβ’ SIMD (Single Instruction Multiple Data) operationβ’ Rotation on plaintext slots:
HEAAN: Summary
π§ = π§1, β¦ , π§ π 2 β¦ π§β² = (π§2, β¦ , π§ π 2, π§1)