Improvements on aggregation and security of encoded data for … · 2020-02-05 · Improvements on...

Improvements on Aggregation and Security of Encoded Data forInformation Transmission, Processing and Reliable Data Storage

by

Juan Camilo Corena Bossa

DissertationSubmitted by Juan Camilo Corena Bossa

In Partial Fulfillment of the Requirements for the Degree of

Doctor of Engineering (Ph.D.)

Supervisor: Prof. Tomoaki Otsuki, Ph.D.

Graduate School of Science and TechnologyKeio UniversitySeptember, 2014

Contents

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Fundamentals of Network Coding . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Attacks on Network Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3.1 Pollution Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3.2 Entropy Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.3 Diversity Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.4 Decodability Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4 Regenerating Codes and Proofs of Data Possession . . . . . . . . . . . . . . . . 121.4.1 Regenerating Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.4.2 Proofs of Data Possession . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.5 Operations on Encoded Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.6 Scope and Contributions of the Dissertation . . . . . . . . . . . . . . . . . . . . 19

2 Pollution Detection in Linear and XOR Network Coding . . . . . . . . . . . . . . . 292.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302.2 Problem Setting and Attack Scenario . . . . . . . . . . . . . . . . . . . . . . . . 312.3 XOR Network Coding Routing Protocols . . . . . . . . . . . . . . . . . . . . . 312.4 Cryptographic Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.4.1 Digital Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.4.2 Blom’s Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.4.3 HMAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.4.4 Cryptographic Accumulators . . . . . . . . . . . . . . . . . . . . . . . . 352.4.5 Related Work on Pollution Prevention for XOR Network Coding . . . . . 36

2.5 Proposal to Identify Misbehaving Nodes . . . . . . . . . . . . . . . . . . . . . . 382.5.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.5.2 Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

ii

2.5.3 Relay Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.5.4 Decision Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.5.5 Security of the Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.6 Proposal to Detect Pollution in XOR Network Coding . . . . . . . . . . . . . . . 412.6.1 Exhaustive Inclusion Strategy . . . . . . . . . . . . . . . . . . . . . . . 422.6.2 Inclusion Strategy for Immediately Decodable Network Coding . . . . . 422.6.3 General Inclusion Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 432.6.4 Security Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.7 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.7.1 Detection Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.7.2 XOR Pollution Prevention . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3 Regenerating Codes and Proofs of Data Possession . . . . . . . . . . . . . . . . . . 533.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.2.1 Regenerating Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.2.2 Proofs of Retrievability . . . . . . . . . . . . . . . . . . . . . . . . . . . 563.2.3 Group Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3.3 Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603.3.1 Unbounded-Use POR for Encoded Files . . . . . . . . . . . . . . . . . . 613.3.2 Bounded-use POR for Encoded Files . . . . . . . . . . . . . . . . . . . 623.3.3 Key Management Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 653.3.4 Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663.3.5 Guaranteeing Decodability in Polynomial Time . . . . . . . . . . . . . . 673.3.6 An Application for Synchronizing Multimedia Collections . . . . . . . . 713.3.7 Improving the Bisection Method . . . . . . . . . . . . . . . . . . . . . . 74

3.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4 Diversity and Decodability Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . 814.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.2 Problem and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834.3 Cryptographic Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4.3.1 Digital Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 834.3.2 Aggregate Message Authentication Codes (MACs) . . . . . . . . . . . . 86

4.4 Existing Work on Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.4.1 Payload-Independent Protocol . . . . . . . . . . . . . . . . . . . . . . . 864.4.2 PDPs with Linear Properties . . . . . . . . . . . . . . . . . . . . . . . . 87

4.5 Proposed Solution to Prevent Diversity Attacks in Multicast Networks . . . . . . 884.5.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.5.2 Parent Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

iii

4.5.3 Processing by N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904.5.4 Parent Approval Generation . . . . . . . . . . . . . . . . . . . . . . . . 904.5.5 Processing at the Children . . . . . . . . . . . . . . . . . . . . . . . . . 904.5.6 Effect of Replacing the Threshold Signature . . . . . . . . . . . . . . . . 914.5.7 Security Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 934.5.8 Probabilistic Alternative to Reduce Overhead . . . . . . . . . . . . . . . 94

4.6 Proposed Solution to Prevent Diversity Attacks in Storage Systems . . . . . . . . 954.6.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964.6.2 Node Regeneration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 964.6.3 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.7 Proposed Solution to Decodability Attacks . . . . . . . . . . . . . . . . . . . . . 974.8 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 994.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

5 Aggregating Encoded Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1045.2 Problem Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055.3 Existing Tools and Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

5.3.1 Searching on Encrypted Data . . . . . . . . . . . . . . . . . . . . . . . . 1065.3.2 Range Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1075.3.3 Research Related to Aggregate Queries . . . . . . . . . . . . . . . . . . 1085.3.4 CryptDB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

5.4 Algorithms with Additive Homomorphism . . . . . . . . . . . . . . . . . . . . . 1105.4.1 Efficient Data Aggregation in WSNs . . . . . . . . . . . . . . . . . . . . 1105.4.2 Paillier Cryptosystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1115.4.3 Secure Addition Using Shamir’s Secret Sharing Scheme . . . . . . . . . 112

5.5 Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.5.1 Scenario Using the Symmetric Scheme . . . . . . . . . . . . . . . . . . 1145.5.2 Scenario Using the Asymmetric Scheme . . . . . . . . . . . . . . . . . . 1165.5.3 Fast Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1165.5.4 Scenario Using Shamir’s Scheme . . . . . . . . . . . . . . . . . . . . . 121

5.6 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1225.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

6 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1306.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Appendix A List of Author’s Publications and Awards . . . . . . . . . . . . . . . . . . 133A.1 Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133A.2 Full Articles on International Conferences Proceedings . . . . . . . . . . . . . . 133A.3 Short Articles on International Conferences Proceedings . . . . . . . . . . . . . 134A.4 Articles on Domestic Conference Proceedings . . . . . . . . . . . . . . . . . . . 134

iv

A.5 Awards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

v

List of Tables

1.1 Encoding operations using bucketization . . . . . . . . . . . . . . . . . . . . . . 191.2 Outline of Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241.3 Outline of Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.4 Outline of Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261.5 Outline of Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.1 Difference between immediate decoding and the general case from decoding per-spective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.2 Possible combinations for a generation of size 4 . . . . . . . . . . . . . . . . . . 442.3 Experiments in C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.4 Execution times in ms for creation using the exhaustive inclusion with 1500 byte

packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492.5 Execution times for creation using the IDNC variant with 1500 byte packets . . . 492.6 Execution times for creation and verification of existing schemes with 1500 byte

packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.1 Times for the pollution detection routine . . . . . . . . . . . . . . . . . . . . . . 763.2 Times for computing unbounded-POR tags for different block sizes . . . . . . . 773.3 Times in ms for computing bounded-POR tags for different field implementations 783.4 Qualitative comparison of the proposed PORs . . . . . . . . . . . . . . . . . . . 78

4.1 Notation summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844.2 Total overhead for N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 914.3 Processing time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 914.4 Different types of overhead associated to different types of signatures at children

nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5.1 Results of randomness of our implementation using the NIST suite of tests 1000times. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5.2 Table structure for the sample application. . . . . . . . . . . . . . . . . . . . . . 1245.3 Comparison of the aggregation schemes. . . . . . . . . . . . . . . . . . . . . . . 1255.4 Table implementing fast aggregation for the sample application. . . . . . . . . . 126

vi

List of Figures

1.1 Linear Network Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Comparison between traditional routing and XOR Network Coding . . . . . . . 51.3 Pollution Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Pollution prevention scheme based on orthogonality . . . . . . . . . . . . . . . 71.5 Cooperative detection scheme for Network Coding . . . . . . . . . . . . . . . . 81.6 Entropy Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.7 Setting for diversity attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.8 Diversity Attack example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.9 Payload-Independent-Protocol (PIP) . . . . . . . . . . . . . . . . . . . . . . . . 111.10 Decodability Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.11 Exact Regenerating Code example . . . . . . . . . . . . . . . . . . . . . . . . . 131.12 A Regenerating Code based on Network Coding . . . . . . . . . . . . . . . . . . 141.13 Load balancing using regenerating codes . . . . . . . . . . . . . . . . . . . . . . 151.14 Diversity Attack in storage networks . . . . . . . . . . . . . . . . . . . . . . . . 161.15 Proof of Data Possession Diagram . . . . . . . . . . . . . . . . . . . . . . . . . 161.16 Configuration of this dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . 201.17 Position in existing research of each chapter . . . . . . . . . . . . . . . . . . . . 211.18 Relation between Chapter 2 and existing research in pollution detection. See Sec-

tion 1.3.1 for further details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.19 Relation between Chapter 3 and existing research in Proofs of Data Possession.

See Section 1.4.2 for further details. . . . . . . . . . . . . . . . . . . . . . . . . 221.20 Relation between Chapter 3 and existing research in Group Testing. See Section

1.4.2 for further details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221.21 Relation between Chapter 4 and existing research in Diversity and Decodability

Attacks. See Section 1.3.3 for further details. . . . . . . . . . . . . . . . . . . . 231.22 Relation between Chapter 5 and existing research in aggregation techniques. See

Section 1.5 for further details. . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1 Example of using elimination to authenticate packets in XOR Network Coding . 322.2 Example of a Merkle tree signature . . . . . . . . . . . . . . . . . . . . . . . . . 342.3 Example of Nyberg’s accumulator . . . . . . . . . . . . . . . . . . . . . . . . . 362.4 Yu et al.’s pollution detection scheme for XOR Network Coding . . . . . . . . . 372.5 Combination step for Yu et al.’s scheme . . . . . . . . . . . . . . . . . . . . . . 372.6 Computing the hash of the i-th row in A (Ai∗). . . . . . . . . . . . . . . . . . . 38

vii

2.7 Packet structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.8 Example without instant decodability . . . . . . . . . . . . . . . . . . . . . . . 422.9 Sample network topology with received packets . . . . . . . . . . . . . . . . . . 43

3.1 A regenerating code based on network coding . . . . . . . . . . . . . . . . . . . 573.2 Storage procedure for a file of length 8 . . . . . . . . . . . . . . . . . . . . . . . 673.3 Twin-code construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693.4 Creating non-systematic nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 703.5 User reconstructing a file from non-systematic nodes . . . . . . . . . . . . . . . 703.6 Number of tests for different configurations of two elements . . . . . . . . . . . 753.7 Simulation for maximum number of aggregate fingerprints . . . . . . . . . . . . 79

4.1 Overview of the proposal to detect diversity attacks in multicast . . . . . . . . . 884.2 Packets sent from parents to N . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.3 Packet sent from N to the neighborhood . . . . . . . . . . . . . . . . . . . . . . 894.4 Total processing time for a network with 4 children. . . . . . . . . . . . . . . . . 1004.5 Total overhead for a network with 4 children. . . . . . . . . . . . . . . . . . . . 101

5.1 Resulting map after applying the hypergeometric distribution . . . . . . . . . . . 1075.2 Sample scenario for the bucketization approach . . . . . . . . . . . . . . . . . . 1095.3 Example of a selection query in CryptDB . . . . . . . . . . . . . . . . . . . . . 1095.4 Steps performed at POS in the symmetric scenario. . . . . . . . . . . . . . . . . 1145.5 Information transmitted to decrypt the aggregate result a. . . . . . . . . . . . . . 1155.6 Diagram of the asymmetric scenario. . . . . . . . . . . . . . . . . . . . . . . . . 1165.7 Diagram of the scenario using Shamir’s Secret Sharing Scheme, for three inde-

pendent storage servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1215.8 Decryption times in our implementation of the three algorithms. . . . . . . . . . 1245.9 Aggregation times in our implementation of the three algorithms. . . . . . . . . . 1255.10 Homomorphic pperations performed VS total rows retrieved . . . . . . . . . . . 126

viii

Improvements on Aggregation and Security of Encoded Data forInformation Transmission, Processing and Reliable Data Storage

Juan Camilo Corena [email protected] University, 2014

Supervisor: Prof. Tomoaki Otsuki, [email protected]

Abstract

The last decade has seen an exciting number of new applications for encoded data, driven byan increasing need to transmit information more efficiently and perform information outsourc-ing. This dissertation is concerned with providing mechanisms to verify the authenticity, integrityand improve performance on operations that occur over encoded data. In particular, we will ex-plore and provide solutions to novel security challenges in several scenarios, namely in a specialtype of routing algorithm known as Network Coding (NC), codes with efficient codeword repairknown as Regenerating Codes, and outsourced encrypted databases performing operations overencrypted data. The goal of this work is to provide humble contributions to enhance the use ofnew technologies such as cloud computing and mobile devices that can access multiple networkssimultaneously.

Chapter 1 presents an introduction to NC, some of its applications and security threats thataffect it. Unlike traditional routing mechanisms, in NC intermediate nodes are allowed to applyencoding operations on the data, forming an erasure code. This is a simple yet powerful idea whoseapplications include: efficient multicast transmission, tolerance to packet loss and improved datastorage. After this, we introduce the concept of a Proof of Data Possession (PDP) which is aprotocol where a verifier can check whether a server is storing a file on its behalf. Then, wedescribe existing mechanisms to guarantee privacy in outsourced databases. The chapter endswith the scope and contributions of this dissertation.

Chapter 2 studies Pollution Attacks in NC and defense strategies to mitigate it. This is atype of attack where the goal of the adversary is to introduce invalid packet combinations in thenetwork. The problem involves creating mechanisms to allow nodes to distinguish between validand invalid codewords in NC. We present a system that can be used when the operation applied bythe intermediate nodes is the exclusive or (XOR) operation. Unlike traditional schemes that workby verifying what properties of the code are preserved during the lifetime of a codeword, we workdirectly over a explicit representation of the packets. This allows us to have lower verificationtimes compared to traditional schemes. In addition, we propose a decentralized system to revokenodes introducing pollution in the network that requires less message exchanges than traditionalsystems involving a central authority.

ix

Chapter 3 deals with performing proofs of data possession on files that have been encodedusing regenerating codes. Regenerating codes are erasure codes used for storage, their main ad-vantage is that they have efficient routines for the reconstruction of missing or polluted codewords.The additional challenge compared to the previous chapter, is that in this one, nodes do not haveaccess to the codewords they are trying to verify. The motivation for this problem arises from thepay-as-you-use model currently in practice along cloud storage services. Owing to the bandwidthcosts, verifying outsourced storage by downloading the outsourced codewords is slower and moreexpensive than performing the test remotely. We present efficient schemes to verify files are avail-able by checking the codewords used to encode them. This is a novel scenario for this kind ofprimitive that usually works over a static file. The solution uses very low bandwidth compared tothe length of the file, and it works in the presence of dishonest servers that might refuse to reportdata loss. Our proposal also works in scenarios where the number of servers is not fixed and newservers are created without contacting the file owner. We achieve this by extending the concept oftraditional proofs of data possession by creating a fingerprint of the file that can be verified evenafter linear transformations of its parts have taken place. This allows to overcome the limitationsof existing schemes using traditional fingerprinting methods. In addition to this protocol, a genericalgorithm to find what exact blocks are missing is presented. The protocol outperforms existingapproaches to find defective codewords in files. Advantages of our protocol come from the ad-ditional information we obtain from the linearity of the fingerprint. Traditional methods to finddefective codewords cannot assume this to improve detection performance.

Chapter 4 focuses on a relatively new security threat to NC called a Diversity Attack. Thisattack occurs when attacker nodes do not perform network coding over all the information theyreceive to decrease throughput. We present solutions to detect this problem with the property thatthe length of the transmission information does not depend on the length of the packets, but ratheron the parameters of network coding for that particular network. Traditional methods to preventthis attack perform the verification at the receivers of the information; for this reason, they musttransmit a larger amount of information to provide evidence of all the nodes participating in theconstruction of a particular packet. Our proposal shifts this responsibility to the nodes contributingto the packet, allowing us to reduce the transmission and computational overhead in the protocol.This chapter also proposes a solution to a new problem in NC called a Decodability Attack whichoccurs in networks where nodes do not buffer packets that have not been decoded. Our solution tothis problem applies concepts from the field of verifiable computing. The goal is to prove that theselection of messages by a sender was appropriate given the information that was available to it.

Chapter 5 deals with performance issues when operations need to be performed on encodeddata. The application scenario is an outsourced expense tracking application where aggregationof encrypted data is performed at a server on behalf of the user. We propose several scenarios toachieve this involving polynomial interpolation and Homomorphic Encryption in the symmetricand asymmetric setting. Homomorphic Encryption is an interesting cryptographic primitive thatallows to perform operations on the data without having to decrypt it first. For the symmetricscenario, we adapt ideas from information aggregation techniques developed for sensor networksto the database scenario. For the asymmetric scenario, we propose an alternative to a technique offast retrieval known as buckets. This technique involves computing a predetermined query as part

x

of the answer arrives. In contrast, our solution answer a larger number of possible queries withthe same amount of work. The motivation for this problem arises from the fact that arithmeticoperations over encrypted data are considerably slower than those performed over plain texts.Therefore this topic is related to the scalability of this kind of applications.

Finally, Chapter 6 summarizes the conclusions and possible venues for future research of thiswork.

xi

Acknowledgments

I would like to express my deepest gratitude to my supervisor Prof. Tomoaki Ohtsuki whosesupport and encouragement made this research possible. His advice was crucial to undertake newresearch challenges and keep persevering even when results were slow to come. I am deeplygrateful for all the empowerment I received under his supervision that let me define my own pace.

The committee members Prof. Iwaso Sasase, Prof. Hiroshi Shigeno and Prof. Fumio Teraokadeserve a special mention, for their precious time and advice that helped improve the quality ofthis dissertation.

My special thanks are extended to the Global Center of Excellence (GCOE) at Keio Uni-versity, for supporting most my stay in Japan for more than 2 years; providing research funding,equipment, research trips to universities overseas; also for invaluable research experiences throughconferences, seminars, winter camps; and for allowing me to meet outstanding people. I wouldalso like to extend my acknowledgements to the administrative staff of this project Maki Adachi,Chinatsu Ichikawa, Mitsuko Watanabe and Ayumi Higuchi, for their support during the length ofthe project.

I am deeply indebted to KDDI R&D Labs Inc. for giving me the opportunity to be a memberof the Information Security Group. To Toshiaki Tanaka, Yutaka Miyake, Shinsaku Kiyomoto,Anirban Basu and other members of the Information Security Group, for their input and supportwhich have made this last year a life changing experience.

Assistance provided by the following organizations was greatly appreciated: Keio LeadingEdge Laboratory (KLL), for offering a research grant to support my trip to IEEE PIMRC 2013.NEC C&C foundation for funding my attendance to IEEE Globecom 2012.

I would like to thank Prof. Chen-Nee Chuah, Han Liu, Mehdi Malboubi, Carlos ColmanMeixner and all the members of Rubynet at UC Davis, for their kindness and insightful discussionsduring my visit. All the members of Ohtsuki Laboratory specially to Jihoon Hong who never letme work alone and whose unlimited energy is a reference for my future endeavors. Naotoshi Yodaand Toshihito Kudo for great experiences and timely help throughout this time. I am particularlygrateful to Oussama Souihli whose advice has had a great impact in the completion of this work.

To my dear friend Oscar Rodrıguez (Yuuji), Daniel Mi compadre, Tania my sister, my fatherEl tigre and my mother whose sacrifices, love and guidance have given me all the opportunities Ienjoy to this day. Last but not least, to Satoko whose beautiful smile can make the coldest winterbe warm like my home on a faraway land.

Juan Camilo Corena Bossa

xii

Chapter 1

Introduction

1

2 CHAPTER 1. INTRODUCTION

This dissertation is concerned with two main topics. The first of them, is providing authen-ticity routines for encoded data for transmission and storage systems performing forward errorcorrection. The second is to provide mechanisms to perform operations on data that have beenencoded for the purpose of security.

1.1 Background

The last decade has seen the development of new applications of encoded data for the purpose ofavailability, redundancy and security. An interesting instance of this tendency comes in the formof Linear Network Coding (NC) [1] which is a technique that has seen diverse uses ranging fromP2P content distribution [2], more efficient wireless routing [3], network tomography [4]. The ideabehind NC is simple yet powerful, it consists of allowing intermediate nodes in a network to treatincoming packets as vectors and applying linear operations over them. By doing this, it is possibleto obtain benefits such as throughput gain, increased availability of information or tolerance topacket loss. Despite its advantages, NC introduces new ways to attack systems using it. The moststudied attack against NC, is the Pollution Attack [5] where nodes do not code correctly pollutingthe information flow. Other attacks are also possible, including: Entropy Attacks [6], DiversityAttacks [7] and a new type of attack called Decodability Attack [8] we proposed.

Another interesting application that has emerged in recent years is Regenerating Codes (RC)[9] which are codes to provide redundancy for storing large volumes of information at severalstorage nodes. Unlike traditional erasure codes that have been available for decades (e.g. Reed-Solomon [10]), RCs are not just concerned with optimizing the amount of storage used by eachstorage node, but also the amount of information that needs to be transmitted to “regenerate” anode that has been damaged. These codes are at the core of storage technologies for massiveonline websites and services such as Facebook [11, 12] and Windows Azure Storage [13]. Owingto its practical relevance and their ability to reduce the storage costs associated to file replicationwithout the high bandwidth requirements for repair of traditional erasure codes, RCs play a crucialrole in cloud storage infrastructure.

Reduction in storage costs at outsourced locations, has made more data that used to be kept lo-cally to be transferred outside the control of their owners. Such concerns have been reflected in theacademic literature in the form of a set of protocols called Proofs of Data Possession (PDP) [14] orProofs of Retrievability (POR) [15], we will refer to them interchangeably throughout the disser-tation. The goal of these primitives is to verify that a file is stored at the remote location efficiently,without transmitting a large amount of data. The motivation for creating such a primitive, is to pre-vent storage service providers from hiding accidental or malicious loss of data from their owners.From a technical perspective, the obvious solution to the problem is to download the file and applya computation locally. Unfortunately, this is not a very efficient strategy for large files, for thatreason, better strategies with significantly less transmission overhead have been designed [16–21].PDPs present interesting challenges similar to those of NC, since regeneration operations in RCscan be seen as an instance of a node performing NC in a transmission network [22]. Comparedto guaranteeing authenticity in NC, PDPs have the additional difficulty that the stored codewordsare not seen by the information owner (verifier) but are performed by the storage server (prover).

1.2. FUNDAMENTALS OF NETWORK CODING 3

Given that the particular combination of codewords stored at a particular server might have neverbe seen by the information owner, guaranteeing the authenticity of regenerated servers presentsinteresting challenges.

Besides transmission and storage, information outsourcing has also fostered demand in theindustry for schemes that can apply operations on encoded data for the purpose of secrecy. Sometechniques that have been applied in practice include: homomorphic encryption [23] where oper-ations over encrypted data can be performed without knowledge of the secret keys; secret shar-ing [24] where information is dispersed securely across different servers; and techniques for fastinformation aggregation [25]. Such technologies can increase the number of applications that canbe outsourced to third parties. Besides security, these technologies present also interesting chal-lenges to make them efficient enough to be used at a massive scale, owing to the loss of efficiencyof performing operations on the encoded domain.

In this dissertation the main focus regarding authenticity of encoded data is to provide so-lutions to security problems arising in NC and PDPs which include: pollution attacks, diversityattacks, decodability attacks; and also, extend current primitives to pinpoint the exact location ofmisbehaving nodes and corrupt nodes. Regarding computations of encoded data, our focus was toincrease efficiency of the proper encoding and decoding routines, but rather on general strategiestreating the primitives as black boxes. This chapter is organized as follows: Section 1.2 intro-duces the fundamentals of NC. Section 1.3 presents existing attacks against NC as well as the newDecodability Attack. Section 1.4 presents existing constructions of regenerating codes and PDPs.Section 1.5 gives an introduction to operations over encoded data. Finally, Section 1.6 providesa summary of the contributions and scope of this dissertation, including the dissertation configu-ration, an outline chapter by chapter of the rearch topics, and relation of each chapter to existingresearch.

1.2 Fundamentals of Network Coding

To perform network coding, packets in the network are treated as vectors, using these vectors,linear combinations are sent instead of the traditional store-and-forward mechanism. Receiverscan decode the original information by reverting the linear combinations applied to the originalpackets. This can be done by using a set of embedded coefficients in every packet to form a globalencoding matrix. The actual procedure for decoding involves finding the inverse of the globalencoding matrix and applying it to the encoded file.

Consider the file [7, 8, 2]T . If we wish to assign each coordinate of the vector to a differentpacket, the file would be embedded in an augmented matrix by prepending the identity matrix asfollows:

⎡

⎢⎣

1 0 0 7

0 1 0 8

0 0 1 2

⎤

⎥⎦ . (1.1)

To transmit a packet, we select a random set of coefficients and apply them to our augmentedmatrix from 1.1. For instance, coefficients

[

4 5 2]

performing operations modulo 11 would


1

3

2

4

5

6

A

A

B

B

αA+ βBαA+ βB

αA+ βB

Figure 1.1: Linear Network Coding in a butterfly topology. Node 3 combines incoming vectors Aand B into a single packet αA + αB, where α,β are coefficients. Nodes 5,6 decode by solvingthe system of linear equations given by the vectors.

result in:

[

4 5 2]

⎡

⎢⎣

1 0 0 7

0 1 0 8

0 0 1 2

⎤

⎥⎦ =

[

4 5 2 6]

. (1.2)

Once enough vectors have been received by the nodes wishing to decode, it is possible to re-cover the original augmented matrix to recover the original information. Continuing the example,assume the received vectors are represented by the following matrix:

⎡

⎢⎣

4 5 2 6

2 9 6 10

3 3 4 9

⎤

⎥⎦ (1.3)

then, the original augmented matrix can be recovered by left multiplying matrix (1.3) by the in-verse of the 3× 3 matrix containing the encoding coefficients.

⎡

⎢⎣

4 5 2

2 9 6

3 3 4

⎤

⎥⎦

−1 ⎡

⎢⎣

4 5 2 6

2 9 6 10

3 3 4 9

⎤

⎥⎦ =

⎡

⎢⎣

6 10 4

7 7 8

4 1 5

⎤

⎥⎦

⎡

⎢⎣

4 5 2 6

2 9 6 10

3 3 4 9

⎤

⎥⎦ =

⎡

⎢⎣

1 0 0 7

0 1 0 8

0 0 1 2

⎤

⎥⎦ (1.4)

In general, to keep messages the same size as the travel the network, the arithmetic is performedusing over a finite field Fq. For this particular example, we set q = 11, but binary finite fields canalso be used.

The usual scenario for network coding involves a source S (or several of them for the multi-source scenario) which makes linear combinations of all the n packets, using random coefficientsα1, . . . ,αn. This guarantees with high probability that the file can be recovered when a sinkreceives n packets, in a process similar to that of fountain codes [26] and other closely relatedcodes [27]. For network transmission, in order to prevent the coefficient matrix from becomingtoo large, large files are separated into generations, only vectors belonging to the same generationare coded together. The size of the generation also has an impact on how fast a node can start todecode the information contained in a single packet.

1.3. ATTACKS ON NETWORK CODING 5

!"

#"

!"

#"

!"

#"

!"

#"

!!#" !!#"

$%&'()*+&,"-*.)+/" 0123*%4"5*'(+/"

"""""""""""""""""""""$(61"

72&%2"""""""""""""""""""""""""""""""""""""""""8(+(9:"

Figure 1.2: Comparison between traditional routing and XOR network coding. Routers can createpackets that benefit more nodes simultaneously by applying XOR operations. In this figure, thenetwork coding approach can complete the transmission in 3 units of time, compared to 4 in thetraditional routing scheme. One node can recover B by computing B = A⊕ (A ⊕ B), the othercan recover A by computing A = B ⊕ (A⊕B).

A special scenario for Network Coding arises when q = 2. Here, the only possible operationallowed among vectors is the exclusive or operation (XOR) denoted by “⊕”. For this reason, thiscase is called XORNetwork Coding. This type of network coding is less computationally intensive,since nodes can recover the information without performing matrix operations. This variant canbe used in constrained environments such as the one presented in Fig. 1.2. There, an intermediatewireless router wants to forward information to nodes that are connected through it. By takinginto account the packets known by each node, it is possible to transmit information to both nodesusing a single packet [3].

For the particular example of Fig. 1.2, the node who transmitted A can recover B by comput-ingB = A⊕(A⊕B). The node who transmittedB can recoverA by computingA = B⊕(A⊕B).In general, if nodes include what packets are known by them in every packet, it is possible to de-sign strategies to encode the packets in a close to optimal way [28]. When nodes are required toonly encode packets that their neighbors can decode in the next hop, we say that the network isusing Immediately Decodable Network Coding (IDNC).

1.3 Attacks on Network Coding

In this section we will cover four types of attacks that affect Network Coding, along with theideas behind existing approaches to mitigate them; shortcomings of existing approaches will alsobe included. Solutions to challenges related to Pollution Attacks will be presented in Chapter 2,whereas challenges related to Diversity and Decodability attacks will be presented in Chapter 4.

1.3.1 Pollution Attacks

Despite its many advantages, using network coding introduces several new points where adver-saries can attack the system. Within these attacks, the most studied one is the Pollution Attack


1

3

2

4

5

6

A

A

B

B

XX

X

Figure 1.3: Pollution attack performed by node 3 in the right network. Vector X cannot be pro-duced as a linear combination of vectors A and B. For this reason, nodes 5 and 6 cannot decodeproperly.

where malicious nodes introduce packets that cannot be produced by any combination of the orig-inal ones. The effect of this conduct is that receivers cannot decode the information. An exampleof this attack is illustrated in Fig. 1.3, where node 3 outputs a vector X which cannot be producedas a valid combination of vectors A and B. The result is that nodes 5 and 6 cannot decode theinformation correctly.

To allow nodes to detect this attack, several constructions exist [29–32]; the common elementin them is an authentication function f(k,m) with linear properties, depending on a secret key kand vectorm, such that:

α · f(k,m) + β · f(k,m′) = f(k,α ·m+ β ·m′) (1.5)

for α,β scalars. Nodes that wish to verify the authenticity of the packets, apply network codingover the vectors and the outputs of f appended to them. Given the linear properties of f , it ispossible to infer the right result for the particular packet that was received. As a concrete exampleof this type of system, we will now explain the idea behind the scheme presented in [29] which isbased on orthogonality among vectors.

The main component from the scheme presented in [29] is that if we have a matrix M andthere is a vector S that belongs to its nullspace, this isMS = [0]. Then, the product of S with anylinear combination of rows ofM will be 0 as well. This scheme starts withM which is a networkcoding matrix of the form [ID] where I is the identity matrix and D is the data that wants to betransmitted. M is then modified by adding an additional column A to form a new network codingmatrix M ′ = [IDA]. Next, a vector S from the nullspace of M ′ is found and given to a nodealong with A. When several nodes need to verify packets, different pairs of vectors A and S mustbe found. Verification involves using A to compute the packet V that corresponds to M ′ given apacket created from the rows of M , and then verifying that its dot product with S is 0. Fig. 1.4has an example for this scheme where D =

[

4 3]T, A =

[

2 1]T, S =

[

2 3 5 3]Tand

operations are performed modulo 7.Schemes based on properties of the network coding matrix like the previous one are efficient

when the networks are small, otherwise the number of vectors that need to be generated increasesconsiderably. A different approach to reduce the number of vectors is to hide the vector usingasymmetric cryptography. However, this approach is very slow in practice to be considered useful.


V = 3 6 2 5

3 6

2

1

= 53 6 2

1 0 4 2

0 1 3 1

1 0 4

0 1 3

Augmented Network Coding Matrix (mod 7)

Network Coding Matrix (mod 7)

Data Coefficients Additional Column

S =

2

3

5

3

Augment the matrix

Find vector in the nullspace

Vector in the nullspace (mod 7)

Find the corresponding

additional secret column

Packet to verify (mod 7)

Create the augmented packet

Generation Stage

Verification Stage

VS = 0

Figure 1.4: Example of pollution prevention scheme based on orthogonality. A vector on thenullspace of an augmented matrix is found, the dot product of any valid packet with the nullspacevector will have 0 as a result.

Another drawback of these schemes is that they are not secure when the finite field where opera-tions are performed is small. This is particularly troublesome in XOR Network Coding where thefinite field only has two elements. For instance, in the scenario from Fig. 1.4, the probability offinding a random packet that is orthogonal to vector S is 1/7. In general, this probability equals1/|F| where |F| is the size of the finite field where operations are being performed.

A different approach called Homomorphic Hashing [33, 34] consists of a special collision-resistant hash function with linear properties. However, this is too slow to be used in practice,given the number of operations needed to compute the function.

In Section 2.6, we present an efficient scheme which can be used in XOR network codingwhose security does not depend on the size of the finite field. We also present a way to reducethe amount of effort needed to implement traditional schemes to identify the nodes introducingpollution in the network. Compared to the existing approach where random elements are sampled[35] and then encrypted, ours achieves better detection probability.

Regarding detecting and isolating the nodes introducing pollution in the network, traditionalschemes usually rely on a Central Authority (CA) to perform the exclusion of the node. Examplesof this methods include [32] where after the detection of a polluted packet, all nodes send infor-mation to the CA stating information about the packets received from each of its neighbors. Usingthis information, the CA can create a graph similar to the one presented in Fig. 1.5 and identifywho the attacker is. In [36], the idea consists in giving each pair of nodes in the network a set ofsecret keys. Assume there is a set of keys k1, k2, k3 and two nodes X and Y . The CA sends asubset of the keys to each node, for instance: assign k1, k2 toX and k2, k3 to Y . It is important tomention that the set of keys given to each node is secret and unique. Every forwarded packet bya node will be signed using these keys. In this case case, Y can know a packet identifies a nodebecause the signature for k2 will match. If the packet happens to be polluted, Y can forward thepacket to the CA which identifies the original sender because of the keys that were used to sign it.The intuition for this system is that without knowledge of the keys of the neighbors, the attacker


1

3

6

2

4

7

5

Figure 1.5: In [32] after pollution is detected, nodes report the packets that have been receivedfrom their neighbors to the CA. Using this information the CA can construct a graph like this one.Here, a red dotted arrow represents the transmission of a polluted packet, a solid black arrow acorrect one. Given the current information it is possible to conclude that the attacker is node 2,since it was the first node that sent a polluted packet.

Time = 1

1

3

2

4

5

6

Time = 2

1

3

2

4

5

6

A

A

B

B

A+B

A+B

A+B

C

D

C

D

2A+ 2B

2A+ 2B

2A+ 2B

Figure 1.6: Entropy Attack performed by node 3. During the first iteration node 3 sends the packetA + B, in the second iteration it sends packet 2A + B. The second packet despite being correctdoes not provide new information to node 4.

must use all its keys, in which case the CA can find its identity. The problem with this kind ofapproach is that the CA must be reachable in order for the system to work. We present a schemein Section 2.5 that does not need an online CA to make other nodes identify the attacker.

1.3.2 Entropy Attacks

There are also less disruptive ways to decrease the throughput when using network coding. Forinstance, Gkantsidis et al. [6] discuss and present a solution for “Entropy Attacks”, which can beseen as a special case of a replay attack. Entropy Attacks occur when a node sends packets thatprovide no new information for the receiver. An instance of this attack is presented in Fig. 1.6where at time = 1, node 3 sends the packet A + B; then at time = 2, node 3 sends the packet2A+ 2B. The attack occurs because the second packet 2A+ 2B can be produced by multiplyingthe first packet A+B by 2, obtaining 2(A+B) = 2A+2B. Given that in linear network codingpackets represent a system of linear equations, packets that are linearly dependent from each otherare not useful to solve the system.

The solution for Entropy Attacks is checking that every received packet is linearly independentfrom the previous ones [6]. If that is the case, then the packet contains new information; otherwise,


S

.

.

.

.

.

.

.

.

.

P1

P|PN |

N ...

C1

C|CN |

.

.

.

.

.

.

D1

.

.

.

Dd

Figure 1.7: Setting for Diversity Attacks in multicast networks. The source S transmits informa-tion intended for nodes Dk, N is any given node in the network performing network coding, Pi arethe parents of N that send information to it. The goal is to allow the children nodes Cj to verifythat information from the parents was used to create the packets transmitted by N.

the packet is discarded. The original scenario considered by the authors for this attack, was a peer-to-peer content distribution network using network coding.

During our research, we did not study different solutions for this problem since existing solu-tions are already efficient enough for practical purposes.

1.3.3 Diversity Attacks

Popa et al. [7] introduced the “Diversity Problem”. In words of the authors, the problem asks thefollowing question: “How can we force a node to code correctly?”. This is defined as coding avalid packet, using random coefficients, over all the packets a node is supposed to receive. Thegeneral setting for the problem is shown in Fig. 1.7. The goal is to allow nodes who receivepackets from node N (N’s children), to verify that N is coding from all the nodes who send packetsto it (N’s parents). An instance of the attack is presented in the right network of Fig. 1.8, wherenode 3 does not forward packet B that was received from node 2. As a result, throughput towardnode 5 is reduced.

This attack is not only relevant to the context of multicast networks, but also for storage sys-tems using network coding such as [9]. The reason for this, is that unlike other erasure codes(such as Reed-Solomon [10]) that can tolerate failures at some nodes, NC can repair failed nodeswithout reconstructing the entire file. However, the ability to recover the file and to regenerate afailed node, depends on nodes performing the combinations using random coefficients from all thenodes sending information to them. This is a special case for the diversity problem, where there isonly one parent (information owner), one child (new storage node) and several intermediate nodes(current storage nodes). In addition, the parent may communicate directly with the child. Since thenew scenario involves a storage system, we want the owner of the information to be able to verifydiversity, without receiving the actual payload. Given that the problem applies to both networkingand storage scenarios, we will refer to the devices in the network either as nodes or servers.

Diversity attacks are different from entropy attacks [6], since the latter arises only when a newreceived packet provides no new information to the receiving node. To illustrate this, considernode 3 in Fig. 1.8. On the left network, node 3 behaves according to the protocol while on theright one, the forwarded packet (A) does not provide new information for node 5 (entropy attack).Using the algorithm proposed in [6] which consists in checking that the newly received packet is


1

3

2

4

5

6

1

3

2

4

5

6

A

A

B

B

A + BA + B

A + B

A

A

B

B

AA

A

Figure 1.8: Diversity Attack performed by node 3 in the right network. Since A is a valid packet,a pollution detection scheme cannot detect misbehavior by node 3; also, the packet provides newinformation to node 4, bypassing successfully the test for entropy presented in [6].

linearly independent from the previous ones, does not give enough information for node 4 to tellapart the scenario on the left from the one on the right. Also, pollution detection routines cannothandle this attack by themselves, because forwarded packets do not pollute the information flow,they just reduce its throughput. In particular for this case, throughput for node 5 is reduced by50%.

The original work dealing with the Diversity Problem [7] presents two constructions whoseoverhead does not depend on the length of the payload of the packets. The first of them is calledPayload-Independent-Protocol (PIP), it consists of appending a token of information from each ofthe parents, in the packet forwarded by N to the children. The size of this token does not dependon the payload, but still allows every child to check the contribution from each parent in the packetsent by node N. An example for 3 parents and one child is presented in Fig. 1.9 where it is shownwhat parts of the original packets are forwarded to the children. In particular, coefficients referto the network coding coefficients, pollution signature refers to the information needed to checkwhether a packet is polluted or not and “From parent to N” is a cryptographic proof that a parentsent the packet to node N. The procedure to use this information to prevent diversity attacks ispresented in detail in Section 4.4.1. Since children nodes are the ones performing the verification,we say this construction is receiver-centered. The second construction called Log-PIP, commitsto the information of the PIP protocol using a Merkle hash tree [37]; then children query N in aprobabilistic fashion.

The problem with the existing construction to prevent diversity attacks, is the amount of pro-cessing that needs to be performed by the children to verify a packet includes information fromall the parents. In addition, the PIP protocol has a considerable transmission overhead as well,even though Log-PIP reduces the transmission overhead, probabilistic testing creates additionalcomplexity in the system. To solve these problems, in Chapter 4 we present a scheme where theparents can check their own contribution to a packet. As a result, we can reduce the amount ofcomputations needed by the nodes in the network. Our proposal takes advantage of the fact that inNetwork Coding, packets can be overheard by the parents as well.

1.3.4 Decodability Attacks

Corena et al. proposed a new type of attack against network coding in [8] called a DecodabilityAttack. This attack consists in finding a subset of packets available to an encoding node, suchthat when encoded together minimize the number of neighbors that can immediately decode thepacket. An example of the attack is presented in Fig. 1.10. There, node 4 outputs a packet


1

2 N C

3

1 1 1

2 2 2

3 3 3

N N N 1 1 2 2 3 3

Coefficients - 32 bytes

Payload - 1428 bytes

Pollution signature – 40 bytes

“From parent to N” - 40 Bytes

# # #

Packet sections

This means packet sent by node “#”

1

3

2

1 2 3

Performs Network Coding and appends parts of the

packets of the parents

N

It verifies the information from the 3 parents and decides

whether N is behaving honestly

Overhead = 112*3 = 336 bytes

Figure 1.9: Payload-Independent-Protocol (PIP) diagram. Node N appends some parts of thepackets received from its parents. Using this information the child node can verify whether all theparents were used or not.

4

1

2

3

5

6

A

A

C

BB

A⊕B ⊕ C

A⊕B ⊕ C

Figure 1.10: Decodability Attack performed by node 4. Packets A,B,C are encoded in a singlepacket A⊕B ⊕ C , this packet cannot be decoded by any neighbor of node 4.

A ⊕ B ⊕ C which cannot be decoded by any of its neighbors. Given that in some systems,nodes must discard packets that cannot be immediately decoded, this reduces network throughput.The reason for discarding packets follows from two reasons, namely the additional complexityinvolving pollution prevention, and creating temporary buffers for storing packets that have notbeen decoded.

We will now present why this attack is different from existing ones. Pollution attacks [30] areequivalent to checking that an encoded vector belongs to the linear span of the network codingmatrix. By the definition of our problem, all the packets are valid combinations of existing correctpackets; hence, it is not a pollution attack. Entropy attacks [6] are equivalent to checking whethera packet is linearly independent from the previous ones. In our scenario, A ⊕ B ⊕ C has newinformation for every single neighbor of node 4. In fact, given that network coding is a linearcode, not being able to decode a packet implies the packet is linearly independent. Therefore,protecting from entropy attacks does not protect against a decodability attack.

In diversity attacks [7], a node must prove a packet was constructed using packets coming froma particular set of nodes. Continuing with Fig. 1.10 shows a situation where a packet is diversebut no nodes can decode the packet immediately. Here, node 4 is coding the packets receivedfrom nodes 1,2,3 (its parents). Despite this, the packet is not useful to any node in the network.


Therefore, protecting against diversity attacks, does not help to prevent throughput loss in theparticular case of nodes 5 and 6.

1.4 Regenerating Codes and Proofs of Data Possession

Up to this point we have discussed applications and security challenges of Network Coding relatedto data transmission. The same idea of allowing intermediate nodes to encode information canbe exploited for storage. In this section, we will describe security challenges and applicationsof applying Network Coding ideas for storage related to Regenerating Codes. These are codesthat can recover nodes in a storage network using less bandwidth than traditional erasure codes.Regarding security, we are interested in allowing a server to prove that it is storing a piece of afile that was encoded using a regenerating code which is a relatively new scenario for proving datapossession. Solutions to the challenges presented in this Section will be covered in Chapter 3.

1.4.1 Regenerating Codes

A Regenerating Code stores a file of length B into n different nodes, each one of them storing αsymbols, such that any k nodes can reconstruct the original file. When a node fails, a replacementnode can be created by connecting to d ≥ k nodes and downloading β ≤ α symbols from eachone of them. dβ is known as the repair-bandwidth. Compared to traditional erasure codes, regen-erating codes use a smaller repair-bandwidth to regenerate a single node. Different constructionsof regenerating codes can focus on either minimizing the repair-bandwidth or storage α.

In the regenerating code literature there are two main approaches, namely functional regenera-tion and exact regeneration. In the first one, even though any k servers are still able to reconstructthe original file, the information of the regenerated node is different from the previous version ofthat node. In the latter, the regenerated node has the same information as before. The code shownin Fig. 1.11 is an example of an exact regenerating code with 4 nodes where 2 nodes are able toreconstruct the file of size 4. The scenario depicted in the figure involves regenerating node 4 us-ing data from the 3 remaining nodes. Unlike traditional erasure codes such as Reed-Solomon [10]that need to download the entire file, in this code only 3 pieces are transmitted. This is possiblethanks to ideas involving Network Coding that allow nodes to output linear combinations of itsstored pieces. This can be seen in the data transmitted by node 2 which is a linear combination ofits two stored pieces.

An example of a functional regenerating code is shown in Fig. 1.12. To regenerate a node, eachcooperating node outputs linear combinations of its data units. In the figure, these values are calleda1, a3 and a4; the replacement node then stores random linear combinations of the received values.The combinations used to produce these values will be linearly independent with overwhelmingprobability when a large finite field is used. As it can be seen from the example, even thoughthe replacement node Cj has the same functionality as the old node N2, its contents are differentfrom the original. One interesting property of this code, is that its method to regenerate a node isequivalent to adding a new node in the network. For this reason, the total number of nodes is notfixed and can grow to match any desired level of redundancy. This is a very interesting propertythat can be used for storage or content distribution.

1.4. REGENERATING CODES AND PROOFS OF DATA POSSESSION 13

m1

m2

m3 m4

m1+m3

m2+m4

m2+m3

m1+m2+m4

Node 1

Node 2

Node 3

Node 4

m2+m4

m1

m3+m4

m2+m3

m1+m2+m4

This is a Network Coding idea

We can regenerate the node directly using

matrix operations

Figure 1.11: Exact Regenerating Code example from [38]. To regenerate node 4, nodes 1 and 3transfer some of the parts of the file they are currently storing, node 2 outputs a linear combinationof them. As a result, a new copy of node 4 is created transmitting only 3 pieces compared to 4with traditional erasure codes such as Reed-Solomon [10].

Given the properties of regenerating codes, it is tempting to use them to balance the load forsites that serve large files. Consider a scenario such as the one depicted in Fig. 1.13, where thereare 3 servers storing encoded pieces of content that is downloaded by the clients. However, someconstraints need to be considered to select the code properly. For example: if the code from Fig.1.12 is used, it is not guaranteed that those downloading the file will be able to decode withoutasking for additional encoded pieces. Other codes such as [39] can guarantee this, but some clientsmust download more than the information they need. We present in Section 3.3.5 an architecturethat overcomes these drawbacks. In addition, the architecture does not require processing at theservers when transmitting files to clients.

Besides multicast networks, we are also concerned about diversity attacks in storage systems.Consider a network where a single parent Pi is connected to a set of storage nodes N1, . . . ,N|N|

where linear combinations of file’s data units will be stored as shown in Fig. 1.14. When one ofthis storage nodes fail, it must be regenerated to keep data available. The regeneration procedureinvolves a group of storage nodes sending random linear combinations of all their stored data unitsto a new node Cj. then Cj stores random combinations of all the received data units 1. Fig. 1.12shows the regeneration of storage node N2 for 4 storage nodes and a file with 4 data units. Notethat even though the length of the file is 4 data units, only 3 data units were transmitted in total. Incase the length of the file is greater than 4 the process can be iterated for groups of 4 elements.

Our goal with storage systems is to provide a procedure to Cj that allows her to know if thestorage nodes are using all the packets stored at them in the transmitted linear combinations. How-ever, unlike the diversity scenario for multicast networks, we can assume there is communicationbetween Pi and Cj, but we want to minimize the amount of exchanged data. To show diversityis an important concern in storage systems, consider the following example: in the initial state inFig. 1.12, any 2 nodes suffice to reconstruct the original file; the same applies when Cj replaces

1Data units are elements of some finite field GF(q). For the finite field GF(28), they are equivalent to bytes.


Node Data unit 1 Data unit 2N1 m1 + 2m2 m3 +m4

N2 m1 +m3 m2 + 2m4

N3 m1 + 5m2 + 3m3 +m4 2m1 +m2 +m3 + 4m4

N4 5m1 + 3m2 +m3 + 2m4 3m1 + 2m2 + 5m3 + 3m4

(a) Initial State for Storage Nodes

Node Value DataN1 a1 m1 + 2m2 +m3 +m4

N3 a3 3m1 + 6m2 + 4m3 + 5m4

N4 a4 8m1 + 5m2 + 6m3 + 5m4

(b) Information Sent by N1, N3 and N4 to create a new node Cj

Coefficients N2’s regenerated data3a1 + 2a3 + a4 17m1 + 23m2 + 17m3 + 18m4

a1 + 2a3 + 3a4 31m1 + 29m2 + 27m3 + 26m4

(c) New node Cj used to regenerate N2

Figure 1.12: A Regenerating Code based on Network Coding [9]. There are 4 storage nodesstoring linear combinations of the original data units M = {m1, . . . ,m4}, any two nodes canreconstruct M and any 3 can create a new storage node. To regenerate a given node, each nodeoutputs a linear combination of its stored data units. In the example, node Cj , receives values a1,a3 and a4 and stores random combinations of them. Even though the regenerated data for Cj isdifferent from the original one, this does not alter the properties of the system; note also that Mwas not reconstructed to regenerate node Cj .

N2. However, let us say a1 becomesm3 +m4 instead ofm1 + 2m2 +m3 +m4 and a3 becomes2m1 + m2 + m3 + 4m4 instead of 3m1 + 6m2 + 4m3 + 5m4. In that case N4 and Cj cannotreconstructM because the resulting system of equations is singular.

1.4.2 Proofs of Data Possession

A Proof of Retrievability (POR) [15] or Proof of Data Possession (PDP) [14], is a proof allowinga computer storing a piece of information to prove the information is still stored entirely [15].The need for this kind of proof arises from the cloud computing scenario where storage serviceproviders might have incentives to hide the loss of rarely accessed data. Somemotivations include:cost efficiency and avoid liability in case of data loss. One simple way for a client to test thedata is available, would be to download it entirely. However, this might be slow and costly dueto the pay-on-demand business model of many providers. An example of this kind of proof ispresented in Fig. 1.15, where the client (verifier) asks a server (prover) to compute a function onthe current version of the file. Then the verifier who knows the answer to the function beforehand,can compare the result to assess whether the server is still storing the file. Note that the amount ofinformation transmitted for the proof namely P and V are significanlty smaller than the file that isbeing checked. In addition, if the server does not have access to the file, it is difficult to computethe right value.

Several PDPs exist in the literature, some properties of the existing schemes include:

1.4. REGENERATING CODES AND PROOFS OF DATA POSSESSION 15

!"#$"#%&! !"#$"#%'! !"#$"#%(!

)*+",-%&! )*+",-%(!)*+",-%'!

Figure 1.13: Scenario with 3 servers each one storing 1/3 of encoded pieces of a file, clients candownload pieces from different servers to obtain the content. All servers handle transmit the sameamount of information regardless of the contents that are accessed by the clients.

Reusable secrets: The same secret can be used several times without compromising its security.The schemes presented in [14,15] and whose description are available in Section 3.2.2 havethis property.

Efficient: Using very lightweight cryptographic primitives such as hash functions [40].

Public verifiability: Anyone not just the owner of the file can verify that the file has been storedat a server [21].

Batch authentication: Several files can be queried simultaneously [18].

Dynamic audits: PDPs that can be updated on files that change through time [19].

Independent verification of multiple replicas of the same file: If several servers are storing acopy of the same file, it should not be possible for a server Si to help another server Sj topass the test, if Sj’s copy is not available. The scheme presented in [20] involves a verifierthat computes a function over the original file once. Then, it encrypts the copy sent toeach server with a different key. Given that the copies at each server look different, serverscannot cooperate among them, despite the fact that they are storing the same file. However,knowledge of the encryption key and the function over the original file, allows the verifierto verify each server independently.

One drawback of existing PDPs is that they were not designed to work for files that wereencoded using a regenerating code. In this scenario, storage nodes can create new nodes that havenever been seen by the owner of the file. This is different from dynamic audits [19] because a third


Pi ...

N1

N|N|

Cj

a1

aN

Figure 1.14: Diversity Attack in the storage scenario. Storage Ni output linear combinations tocreate a new storage node Cj, by selecting coefficients carefully, it is possible that Cj does not holduseful information. Unlike diversity attacks for transmission, here the source of information hasaccess to node Cj directly. However, for efficiency purposes Pi does not want to download theinformation in it.

m1

m2

m3

m4

Compute F with this V Here is my P’

Encrypted V,P

I need the proof

V,P

P = P’, the file is still there

Verifier Prover

Figure 1.15: Diagram of a one-time use Proof of Data Possession. Before deleting the local copyof a fileM = m1, . . . ,m4, the verifier computes a function F (M,V ) depending on a secret valueV and the file M , the result of this function is called P . V, P are encrypted and stored at theserver along with the file. During verification, V is revealed to the to the prover. If the result P ′ ofapplying F (V,M ′) at the server matches the previous value P , then the file is stored at the server.HereM ′ is the current version of the file stored at the server.

party is modifying the file; it is also different from the independent verification scenario in [20],because the owner of the file has seen all the different encrypted versions of the same file which isnot the case here. PDPs for regenerating codes are similar to detecting pollution in network codingin the multicast scenario with the additional constraint that the packet that needs to be verified isstored at another node. This approach was taken in [38] where a pollution detection algorithmcalled SpaceMAC is adapted to verify encoded files remotely. Our constructions in Section 3.3.1and Section 3.3.2 achieve this same goal.

Another drawback of existing PDPs, is that they only provide a binary answer: either all theblocks of a file that were tested are correct or at least one of them is not. However, no informationis given regarding which blocks are the incorrect ones. We address this problem in Section 3.3.7where we propose a way to find all the incorrect blocks of a file while reducing the amount of invo-cations to the verification routine of PDPs. This falls into the realm of a branch of Combinatoricsknown as Group Testing [41].

1.5. OPERATIONS ON ENCODED DATA 17

Group Testing

The idea of group testing is that less tests can be performed if several elements are groupedtogether, instead of testing elements individually. There are mainly two types of group testingnamely, Adaptive and Nonadaptive. Their difference is that in the nonadaptive case, the groupsof elements that will be tested together are settled before the outcome of the tests; in the adaptivescenario, it is possible to change the strategy based on the outcome of the previous tests. The firstgroup testing method [41] is an adaptive test that falls into the category of Sequential Algorithmswhere the number of defective elements is small. Since grouping many elements together is likelyto give a result that many of them are correct using a single test, these algorithms focus in findingthe optimal size of the groups. Since in a cloud environment we cannot make any assumptionsover how many blocks are incorrect, we are interested in Competitive Group Testing where thenumber of defective elements is not bounded. Examples of this algorithms include [42] (Chapter4):

• Bisection: Split elements in two halves, test further if there are incorrect elements in any ofthe halves.

• Doubling: Increase the size of the groups exponentially until an error is found. For instance:test groups of size 1, 2, 4, 8, . . . 2n.

• Digging: Apply special algorithms to find one defective element in a group efficiently.

Nonadaptive algorithms are usually based on binary codes such as superimposed codes [43].These codes have the property that the union of the groups representing of up to d different tests,does not contain all the elements of a group that is not in the union.

1.5 Operations on Encoded Data

Even though Network Coding offers a very interesting realm of applications, it is not the onlytechnology where operations are applied on encoded information. One interesting applicationcomes from the database literature, where a user wants to outsource the database to a third partythat is not fully trusted. To guarantee security, the information must be encoded in a way thatallows the third party to perform operations on the data without actually knowing its value. Anencoding function satisfying this requirement should have the following property:

E(k,m1 ⊕m2) = E(k,m1)⊗E(k,m2). (1.6)

Here E is the encoding function which depends on a secret value k and operates on a plaintextvalue m. Operations ⊗ and ⊕ are two operations where the property holds. In particular for ourscenario, we are interested in encoding functions where the ⊕ = +, here + denotes addition overthe integers. These functions allow addition operations which are common in accounting softwarewhere a user might be interested in computing revenues and expenditures in a given period of time.

There are several encoding functions in the literature satisfying this property. For instance:if several servers are available, it is possible for a user wishing to encode a secret number a0 to


create a polynomial having a0 as a free term:

f(x) = anxn + . . .+ a1x+ a0. (1.7)

given n+1 pairs of the form (f(i), i) it is possible to recover a0. However, having n or less pairsdoes not provide any information about a0, this is the principle behind Shamir’s Secret SharingScheme [44]. Given another polynomial:

g(x) = bnxn + . . . + b1x+ b0, (1.8)

then having n + 1 pairs of the form (f(i) + g(i), i) allows us to recover a0 + b0 in a secret way.To add a new record to a database, a user has to create a new polynomial to hide the free term andsend to the i-th server the image of the polynomial at i. When asked to perform addition, serverssimply add the numbers as if no encoding has been performed. This system is computationallylight, but its problem is that several servers are needed to store the database.

For single-server systems it is necessary to use encryption. One interesting algorithm by Pail-lier [23] has the following property:

E(k,m1 +m2 mod N) = E(k,m1) ∗ E(k,m2) mod N2 (1.9)

hereN is a large number product of two primes, + and ∗ are the usual addition and multiplicationof integers respectively, the mod operation implies taking the residue of the division byN . Thisalgorithm works for a single server and can be used to add integers as the previous one. Anotherinteresting property is that no information about k is necessary to perform the homomorphic op-eration. However, at the time of the writing N should be at least a 2048-bit number. This makesadditions significantly slower than in the polynomial-based scheme. To provide a concrete exam-ple, the overhead between adding 106 numbers encrypted using 2048-bit Paillier and the additionof the same number of plain 64-bit numbers in Java 2, was about 3796 times slower.

Given the different advantages and drawbacks of the existing technologies that can be used toperform computations over encoded data, it is important to design ways to use them efficiently, sothey can have a more significant impact in practice. In Chapter 5 we focus on how to minimizethe number of additions that need to be performed to compute a result. To show this is possible,consider the scheme from [45] shown in Table 1.1.a where each record in a table represents theincome for a given month. A strategy to perform less operations is presented in Table 1.1.b wherebuckets are created in advance with aggregate results of several records. If the incoming querymatches one of the buckets such as January to March or April to June in the example, then lessoperations are needed to return the right result. However, if the buckets do not fit the query exactlyadditional operations need to be performed. In Section 5.5.3 we present a replacement to thebucket strategy where any query that returns continuous rows can be answered in constant time.We achieve this by precomputing the sum from the initial part of the table and then subtracting thepart of the table that is not needed.

2JDK 1.8 in a Mac OS X using an Intel Core 2 Duo clocked at 2.53 GhZ. Big number arithmetic was implementedusing the default BigInteger class.

1.6. SCOPE AND CONTRIBUTIONS OF THE DISSERTATION 19

Month IncomeJanuary 10000February 5000March 15000April 20000May 30000June 25000

(a) Income per month

Month IncomeJanuary to March 30000April to June 75000

(b) Buckets containing income for 3months

Table 1.1: Encoding operations using bucketization

In Chapter 5 of this dissertation, strategies to optimize the number of operations are presentedfor the multi-server and single-server scenarios. Our proposal also considers a proposal that re-duces the amount of computation needed by the user. This is important in the studied settingbecause user resources are likely to be limited.

1.6 Scope and Contributions of the Dissertation

This dissertation consists of six chapters. In Chapters 2, 3, 4 and 5 novel techniques are presentedto address different problems related to encoded data such as authenticity and performing efficientaggregate operations over it. Each chapter contains the particular problem description, the relevantexisting literature, the proposed methods and their evaluation. The outline of this dissertation issummarized in Fig. 1.16.

The core component of this dissertation revolves around attacks on Network Coding. Thistopic is distributed along Chapter 2 that considers pollution attacks, Chapter 3 that considers Proofsof Data Possession over encoded files, and Chapter 4 that develops solutions for Diversity andDecodability attacks in Network Coding. The problems in chapters 2 and 3 are tightly related,they both consist in finding whether a piece of information is a valid codeword of an erasure codewith special properties. The difference in both scenarios is that in Chapter 3, the verifier does nothave access to the codeword because it is stored at a different location. Even further, RegeneratingCodes can be seen as a special case of Network Coding. Chapter 5 addresses the problem ofperforming data aggregation efficiently in an outsourced file that has been encoded for security.Even though Network Coding and other regenerating codes could be used in this scenario, wefocused on schemes that can offer greater security. Finally, Chapter 6 contains the conclusions ofthis dissertation. The relation among chapters of this dissertation can be found in Fig. 1.17.

The relation among the solutions in each chapter and existing research is presented in sep-arate figures. Fig. 1.18 contains the relationship of Chapter 2 with existing research and ourcontributions which include a pollution detection scheme for XOR NC based on cryptographicaccumulators, and a decentralized scheme to detect and isolate misbehaving nodes. Chapter 3contains two figures: Fig. 1.19 relating our linear fingerprint constructions for PDPs over regener-ating codes, within the area of PDPs; and Fig. 1.20 showing the relationship with algorithms usedin Group Testing [41]. Fig. 1.21, presents how our sender-centered approach fits in the context ofexisting prevention schemes for diversity attacks in Chapter 4. Fig. 1.22 presents our alternativeto buckets for efficient aggregation of encoded information in Chapter 5.


!"#$%&'()*((+,%'-./01-,(

!"#$%&'(2*((!-,03/45-,(#,.(6/%/'&(7-'8(

!"#$%&'(9*(:-33/1-,(;&%&01-,(5,(<5,&#'(#,.(=>?(@&%A-'8(!-.5,B(

•  C0"&D&(%-(5.&,1EF(0#/45,B($-33/1-,G(•  :-33/1-,( .&%&01-,( 40"&D&( E-'( =>?( @&%A-'8(

!-.5,BG(

!"#$%&'(H*(?&B&,&'#1,B(!-.&4(#,.(:'--E4(-E(;#%#(:-44&445-,(

•  9(40"&D&4(%-($'-I&(&,0-.&.(.#%#($-44&445-,G(•  J,(#3B-'5%"D(%-(K,.(#33(%"&(D5445,B(-/%4-/'0&.(

L3-084(&M05&,%3FG(•  J'0"5%&0%/'&(E-'(#(0-,%&,%(.54%'5L/1-,(,&%A-'8(

L#4&.(-,('&B&,&'#1,B(0-.&4G(

!"#$%&'(N*(;5I&'45%F(#,.(;&0-.#L535%F(JO#084(•  C0"&D&(%-(.&%&0%(;5I&'45%F(JO#084G(•  +,%'-./01-,( -E( ;&0-.#L535%F( JO#084( #,.( #(

40"&D&(%-(.&%&0%(5%G(

!"#$%&'(P*(JBB'&B#1,B(Q,0-.&.(;#%#(•  J'0"5%&0%/'&4( E-'( &M05&,%( -/%4-/'0&.(

#BB'&B#1-,(-E(&,0-.&.(5,E-'D#1-,(L#4&.(-,( $-3F,-D5#3( 5,%&'$-3#1-,R( 4FDD&%'50("-D-D-'$"50(&,0'F$1-,(#,.(#4FDD&%'50("-D-D-'$"50(&,0'F$1-,G(

Figure 1.16: Configuration of this dissertation

An outline of the research problems in each chapter, conventional approaches and their limi-tations as well as our proposed solutions and results are presented in Tables 1.2, 1.3, 1.4, 1.5.


Network Coding Proofs of Data

Possession

Pollution Attacks

Chapter 2

Diversity Attacks

Decodability Attacks

Chapter 3

Operations over Encoded Data

Aggregation of Outsourced Encoded Data

Transmission

Encoded Data

Security

Storage

Content Distribution

Regenerating Codes

Chapter 4

Chapter 5

Figure 1.17: Position in existing research of each chapter

!"##$%"&'()*)+%"&'

,&-.*".,&-'/)012+3%"&'

41&)30'5$*6)&%+3*"07'

8$##793+)'

/)012+3%"&':;<='

,00"0'

>"00)+%"&':?@='

A03-1%"&3#'

5$*6)&%+3*"07':BC='

D"E'F93+)':G?='

8"&.41&)30'

5$*6)&%+3*"07'

H"I"I"0961+'

H3761&J':GK=' >0L9*"J03961+'

5++$I$#3*"07'8)*E"0M'

>"-1&J'F1J&3*$0)7':G@='

NII)-13*)'/)012+3%"&'

F)+%"&';OB'

P$#%+37*'5$*6)&%+3%"&'

F1J&3*$0)'

5I"0%Q3%"&'

:CG='

P)773J)'

5$*6)&%+3%"&'>"-)7'

:RB='

(17%##3%"&'

>"-)7':C?='

HL9"*6)717'A)7%&J'

:SB=''

P17T)63U1"0'()*)+%"&'

N&'8)*E"0M'>"-1&J'D"E'F93+)'

F)+%"&';OR'

Figure 1.18: Relation between Chapter 2 and existing research in pollution detection. See Section1.3.1 for further details.


!"##$%&#$&'()(&!#%%*%%+#,&

-#.,/*/01%*&1,2#.,/*/01%*&

3/4*"%("+(5&6""#"&7#""*89,:&;<=>&

3.)?*,98()*/&6,8"@A9#,&;BC>&

D.+)(25*&$#"&&6E(8)&7#A+*%&

D.+)(25*&$#"&&6,8#/*/&7#A+*%&

!%*./#0F(,/#G&H.,89#,%&

'+%8"*)*&&I#:("+)?G%&;<B>&

F(,/#G&&'#)0!"#/.8)%&

I+,*("&3.)?*,98()#"%&

D.+)(25*&$#"&&6,8#/*/&7#A+*%&

D.+)(25*&$#"&&6E(8)&7#A+*%&

6E)*,/*/&D?(8?(G0J()*"%&

D?(8?(G&J()*"%&;<K>&

D*89#,&LMLM<&

D*89#,&LMLMN&

Figure 1.19: Relation between Chapter 3 and existing research in Proofs of Data Possession. SeeSection 1.4.2 for further details.

!"#$%&'()*+,&

-#+./01%*2(&& -#+./01%*2(&&

314"56&&

7()5,+)&89:;& <(=$(+*1>&

/>,#"54?@)&

A#@%(**2(&&

/>,#"54?@)&

B5)(C*#+&89D;& 7#$E>5+,&89D;& 75,,5+,&89D;&

F#@#@#"%?5C&

B5)(C*#+&<(C*#+&GHGHI&

Figure 1.20: Relation between Chapter 3 and existing research in Group Testing. See Section1.4.2 for further details.


!"#"$%"&'(")*"&"+,-..&/0#1"2, 3")+"&'(")*"&"+,-..&/0#1"2,

4"*"&5$)$26#,

4"*"#6/),

7089/0+,:)+".")+")*,

7&/*/#/9,;7:7<,=>?,

7&/@0@$9$26#,

4"*"#6/),

A/B0&$*15$#,7089/0+,,

:)+".")+")*,7&/*/#/9,;A/B'7:7<,=>?,

4"*"&5$)$26#,

4"*"#6/),

-..&/%09'C02"+,

-..&/0#1,

4"#/+0@$9$*8,-D0#E,

4"*"#6/),7&/*/#/9,

4"*"#6/),7&/*/#/92,F/&,4$%"&2$*8,

-)+,4"#/+0@$9$*8,-D0#E2,

3"#6/)2,GHI,

0)+,GHJ,

3"#6/),GH>,

Figure 1.21: Relation between Chapter 4 and existing research in Diversity and DecodabilityAttacks. See Section 1.3.3 for further details.

!""#$"%&'()*$+,(-./$0)

1-("2$)1$#3$#) 4/2&52$)1$#3$#)

6#-3%7$)8$9) 6/:2-+)8$9)

;/+<$70)=>?@) A9(%B-+)6#'"#%BB-(")

1$+#$7)1,%#-(")=??@)

4%22$%:2$))C(+#95&'()=>D@)

1$+&'()DEDEF)

C2$B$(7):9)$2$B$(7)%GG-&'())=>F@)

Figure 1.22: Relation between Chapter 5 and existing research in aggregation techniques. SeeSection 1.5 for further details.


Table 1.2: Outline of Chapter 2

Research Problems

• Identify Malicious Nodes in Network Coding.

• Design mechanisms to prevent pollution attacks in XOR NC.

Conventional Approach

• In [29–31] properties of the network coding matrix are used. Forthe case of XORNC, in [35] random positions are sampled, XORedtogether and encrypted.

• In [32], nodes notify a controller stating if they have seen a pollutedpacket, the Central Authority (CA) finds the attacker based on thisinformation. Similarly, in [36] nodes are assigned some shared se-crets with their neighbors and the CA. When pollution is detected,the CA is notified and proof is sent.

Limitations

• In the existing cooperative schemes for detection of maliciousnodes, the central authority needs to be contacted to notify honestnodes that a malicious node has been expelled from the network.

• Existing approaches to prevent pollution attacks either sample ran-dom positions in the payload and increase the size of the packetconsiderably [35], or require computing the authentication functionmany times [29–31] because their security depends on the size ofthe finite field used for NC.

Proposed Solution

• We provide in [46], a decentralized mechanism for node to detectand exclude malicious nodes by revealing information locally thatallows any honest node in a neighborhood to exclude a maliciousnode.

• We show in [47] a mechanism based on Cryptographic Accumula-tors [48] whose security does not depend on the size of the finitefield used for NC.

Summary of Results

• A system able to detect and exclude malicious nodes for NetworkCoding without the intervention of a central authority.

• A scheme suitable to protect XORNC that keeps overhead constantand that requires a single evaluation from the nodes relaying infor-mation. The system works for small generation sizes, and has atmost logarithmic overhead on the number of retransmissions.



Research Problems

• Design Proofs of Data Possession for data that have been encodedusing erasure codes.

• Prevent Diversity Attacks in storage systems using Functional Re-generating Codes.

• Improve existing methods to find the defective elements usingProofs of Data Possession.


• In [14, 15, 49] proofs of data possession for an exact copy of a filehave been developed. Their approach consists in hiding the authen-tication tokens using cryptographic techniques.

• Regarding diversity attacks in storage, it is possible to use existingtechniques for diversity [7].

• Group Testing algorithms [50] can be used to find the defectiveelements in a server.

Limitations

• Most PDP constructions do not work for files where a functionalregenerating code was applied. Those considering them [51], donot consider pollution attacks during the regeneration stage.

• Traditional methods for diversity, assume the originators cannotcommunicate with the nodes who receive the information. We candesign better methods by lifting this restriction.

Proposed Solution

• We provide in [52], a PDP based on the computation of a dot prod-uct of the file with a series of secret vectors. The linear propertiesfunction allows us to use it for encoded files.

• We provide in [52], an improvement bisection method to find ele-ments that differ between local and remote storage. We extend thisresult in general for any PDP scheme.

Summary of Results

• A PDP scheme that can be used for files encoded with functionalregenerating codes.

• An algorithm for finding defective elements in remote storage thatoutperforms the bisection method.



Research Problems

• Prevent Diversity Attacks in Network Coding for Data Transmis-sion in Linear Network Coding.

• Prevent Decodability Attacks in XOR Network Coding.


• In [7] the authors propose a mechanism where children of a nodecan verify the contribution of each node in the neighborhood.

• For Decodability Attacks no conventional approach exists. We areto the best of our knowledge the first to work on this problem.

Limitations

• Existing schemes to prevent diversity attacks do not work well withdynamic topologies. They also require nodes in the network to per-form a number of computations proportional to the number of nodesin their vecinity.

Proposed Solution

• We provide in [53], a mechanism where the topology can be dy-namic. The system also reduces computation by making each nodeverify only its own contribution to a given packet. Once every gen-eration, an aggregate computation is performed to assess the hon-esty of a given node.

• We show how our solution can be adapted to prevent decodabilityattacks.

Summary of Results

• A system who is able to prevent diversity attacks, that requires onlyto check the contribution of one node to a given packet, comparedto the entire neighborhood for existing schemes.

• A modified version of our scheme that can be used to prevent de-codability attacks in XOR network coding.



Research Problems

• Improve the efficiency of additions over encoded data.


• Precompute the result of some functions over a range of values andstore the results in buckets [45]. These results can be used later toretrieve the results more efficiently.

• Use a secret sharing schemes such as Shamir’s [44] to divide theencoded data in several servers. The information is then aggregatedat the client.

Limitations

• Schemes based on buckets incur in increased costs, when the sizeof the bucket does not match the query exactly.

• Systems based on Secret Sharing require more than one server toguarantee security.

Proposed Solution

• In [54] we propose several architectures for a single server to per-form aggregation queries. The first of them is based on a malleablesymmetric encryption scheme from [25]. The second is based ona public key homomorphic scheme such as Paillier [23]. Bothschemes are based on dynamic programming techniques.

Summary of Results

• Our techniques can perform aggregation queries by performing23.3% less operations than existing approaches.

• We derived an non-tight upper bound of 44% on the number of ho-momorphic invocations needed by our scheme for any combinationof retrieved rows from the database.

Chapter 2

Pollution Detection in Linear and XORNetwork Coding

29

30 CHAPTER 2. POLLUTION DETECTION IN LINEAR AND XOR NETWORK CODING

2.1 Introduction

The problem of detecting misbehavior in network coding has received significant attention inacademic literature [29, 55]. The so called “Pollution Attack” [5] described in Section 1.3.1 ,where a misbehaving node (attacker) injects packets in the network, is one of the problems relatedto this technology that has received more attention. Despite this, there are still problems that needto be solved in this matter, such as detecting and isolating the attackers efficiently and providingefficient mechanisms when using network coding in small finite fields. For the latter, XOR networkcoding is important due to its fast decoding, which is useful in constrained environments.

One crucial step needed to exclude attackers from the network, is to provide an efficient wayfor honest nodes to authenticate other nodes. Otherwise, attackers could easily impersonate honestnodes to avoid detection. This problem is known as Broadcast Authentication.

In this chapter, we present a mechanism that allows honest nodes to exclude misbehavingnodes, by not forwarding packets coming from these nodes in the future. To achieve detection,we base our scheme on the security properties of systems aimed at detecting pollution in networkcoding. For the particular case of XOR network coding, we design our own pollution detectionscheme based on cryptographic accumulators. To provide authentication, we present a fast authen-tication scheme based on Blom’s key distribution scheme [56]. Our construction guarantees eitherthat the sender of a packet is identified, or an honest node does not waste significant resourceschecking a packet whose sender cannot be identified.

Unlike other constructions in the literature [32, 36] where a central authority is needed toexclude a misbehaving node, our construction allows any honest node to take proper action againstattackers without contacting the network’s central authority. Since it is based on cryptographicprimitives with provable security properties, its security is high. Our proposal is also reasonablein terms of assumptions, we do not require knowledge of the topology beyond neighboring nodes,making it suitable for dynamic networks. The drawback of our proposal is linear transmissionoverhead in the number of neighbors a given node has. Another not so significant drawback isthat the proposal becomes vulnerable when more than c attackers collude. However, this doesnot represent a significant challenge since the collusion bound can be set arbitrarily large withoutsignificant impact in performance.

Even though our identification proposal can be used in networks where network coding is notused, it is particularly relevant for network coding because pollution detection routines based onhomomorphic digital signatures [30] are computationally intensive. Without proper identificationof attackers, node resources can be exhausted by simply flooding the nodes with malformed pack-ets. To circumvent this bottleneck, protocols such as [55, 57] use time as a source of asymmetry;however, this requires loose synchronization among all nodes in the network and packet buffer-ing. These methods can also benefit from our proposal, because information from neighbors canbe used to exclude attackers faster. Other protocols such as [29] are based on a very clever ob-servation; despite packet combinations, the linear span of the original vectors does not change;nevertheless, in order to turn this into a system that can withstand byzantine attackers, either asource of asymmetry or a key distribution scheme is necessary, such as the used in [31]. For thecase of XOR network coding our pollution detection scheme can be used efficiently for smallgenerations.

2.2. PROBLEM SETTING AND ATTACK SCENARIO 31

The rest of this chapter is structured as follows: in Section 2.2, we introduce our problemsetting to identify misbehaving nodes. Section 2.3 introduces the routing mechanisms of XORnetwork coding, this will illustrate the motivation for the the different pollution prevention pro-posals introduced in this chapter. In Section 2.4 we present the cryptographic primitives needed forboth proposals. Section 2.5 presents our general proposal to detect an isolate misbehaving nodes.Section 2.6 shows our proposal to detect pollution in XOR network coding. Finally Sections 2.7and 2.8 present the experimental results and conclusions respectively.

2.2 Problem Setting and Attack Scenario

There is a wireless network N with |N | = n nodes and a set of incorruptible sources that wish tosend information using network coding. Nodes in the network can be compromised by an attacker,which can control all the resources associated to those nodes. Our goal is to identify the attackersintroducing pollution in the network and isolate them. We assume the existence of a pollutiondetection primitive such as: [29, 30, 55] or the one presented in Section 2.6 for the case of XORnetwork coding.

We define the security requirement for the MAC based system used in the identification rou-tine, in terms of a security game proposed in [58]. The desired security property states that noprobabilistic polynomial-time adversary has a positive expected payoff when playing the follow-ing game: An attacker can ask to receive the output of the MACs on a sequence of messages{m1, . . . ,mg} of its choice, and then decides to quit or to gamble. If it quits it receives a paymentof 0. Otherwise, it chooses a messagem /∈ {m1, . . . ,mg} and tries to guess the value of the MAConm. The adversary receives (1− (1/q)) if his guess is correct, and pays 1/q otherwise.

Intuitively what we want from the security definition, is that our MAC construction cannot beforged by an adversary, even after being exposed to outputs of the MAC function several times.We will assume that nodes can identify their current neighbors before broadcasting. A map of theneighborhood can be created by verifying the sender of the packets received by a given node. Incase the map of neighbors is not complete, nodes that do not appear in the mapping can verifypollution, but at a higher computational cost. Our final assumption is that there are no more thanc compromised nodes during the lifetime of the network.

2.3 XOR Network Coding Routing Protocols

Unlike linear network coding over larger fields where each packet is a linear combination of allthe packets to be transmitted, XOR network coding takes a different approach based on codingopportunities. Informally speaking, a coding opportunity arises when a node can deliver informa-tion simultaneously to more than one node by transmitting a single packet. In Fig. 2.1, there isa coding opportunity for node 3, because node 4 wants a packet node 5 has, while node 5 has apacket node 4 wants. Therefore, by computing the XOR of the two packets, a packet that providesnew information to both nodes simultaneously is produced. A protocol wishing to take advantageof these opportunities must include a way for nodes to transmit the set of packets each node “has”and “wants”. Since nodes receiving encoded packets can decode immediately, this type of network


1

3

2

4

5

(1,0,0,0)

(1,0,0,0)

(0,0,1,1)

(0,0,1,1)

(1, 0, 1, 1)

(1, 0, 1, 1)

Authenticated Packets(0, 1, 0, 0) (0, 0, 0, 1)(1,0,0,0) (0, 0, 1, 0)(1, 1, 0, 0) (0,0,1,1)

Non-Authenticated Packets(1, 0, 1, 1) = (1,0,0,0) ⊕ (0,0,1,1)

Figure 2.1: XOR-network-coding network with a generation of size n = 4. There is a set ofauthenticated packets of size 6, which can be used to authenticate packets that are not in thelist, by performing XOR (⊕) elimination on the received packets. For instance: Node 4 canverify (1, 0, 1, 1) by verifying (1, 0, 0, 0) is in the list of authenticated packets and then computing(0, 0, 1, 1) = (1, 0, 1, 1) ⊕ (1, 0, 0, 0); now, the result can be found in the list of authenticatedpackets.

coding is known as Immediately Decodable Network Coding (IDNC). Fig. 2.1 also show a rudi-mentary pollution prevention scheme which includes a list of previously authenticated packets bythe source that is sent to the nodes. Using this list, nodes can verify whether a packet is pollutedor not. We will now describe mathematically what a coding opportunity is and then explain theintuition of the proposal from section 2.6 as a refinement to this approach.

In mathematical terms, a coding opportunity can be defined using a graph G. Vertices of Gare labeled as vi,j where i is a node in the network and j is a packet of the current generation.A vertex is created only when node i wants packet j [28]. For instance: before transmitting anypackets, the graph corresponing to Fig. 2.1 for a generation of size 8, would have 40 = 5 · 8vertices v1,1, . . . , v1,8, . . . , v5,1, . . . , v5,8. Two vertices vi,j, vk,l ∈ G are connected when any ofthe following situations occur:

1. If j = l: this means node i and node k want the same packet.

2. If node i wants packet l and node k wants packet j: one node has a packet the other wantsand vice versa.

In the previous graph, every subgraph where all vertices are connected (clique) is a coding oppor-tunity. After finding a clique in the graph, all the packets involved in the set of vertices formingthe clique are XORed together. This construction guarantees that all the nodes which appear inthe clique can decode the packet immediately after reception. In terms of overall network perfor-mance, it is desirable to select the cliques with the largest number of nodes, because that benefitsthe largest number of nodes. By analizing the properties of the cliques of this graph, it is possiblefor the source to select packet combinations that are likely to occur during transmission.

Even though finding cliques in graphs is an NP-Hard problem in the general case, for thecontext of XOR network coding, it is possible to find cliques that improve network throughputusing strategies such as: retransmitting the packet wanted by most nodes [59]; selecting a packetwanted by at least one node at random, then combining it with other packets as long as the resultingpacket can be decoded immediately by at least a given number of neighbors on the next hop [3];

2.4. CRYPTOGRAPHIC PRELIMINARIES 33

or exhaustive search with smart prunning [28]. Traditional graph algorithms such as [60] can alsobe used.

Our proposal to prevent pollution in XOR network coding is based on several observationsfrom the routing protocols as well as the size of the generation used to transmit packets. As itcan be seen in Fig. 2.1 there is a list of authenticated packets, once the list has been sent to thenetwork the nodes can simply verify whether the packet that was received is in the list or not.The problem is then how to make the list small enough to make the protocol practical. This canbe done using a cryptographic structure known as a Cryptographic Accumulator that is explainedin Section 2.4.4. This structure can represent all the packets in the list in a very compact way,allowing nodes to verify whether the received packet is authentic or not. However, it is hard for anadversary to make nodes believe that a fake packet is part of the accumulator. In Section 2.6 wepresent several strategies to reduce the amount of elements added to the accumulator dependingon network parameters.

2.4 Cryptographic Preliminaries

In this section we will present some existing authentication mechanisms, along with a key assign-ment needed for our proposal.

2.4.1 Digital Signatures

Digital signatures allow any node in possession of a public key P , check that a node in possessionof the corresponding private key S, generated amessagem. However, the knowledge of P does notallow nodes to produce valid digital signatures. To achieve this property, some kind of asymmetrybetween signers and verifiers is needed.

In the RSA signature scheme [61], asymmetry comes from number theory. Compute n = pq

(public key), where p, q are prime numbers of a suitable size. Then, compute φ(n) = (p−1)(q−1).Select a number e relatively prime to φ(n) and find its inverse modulo φ(n). To sign a message0 < m < n, compute the signature as σ = md mod n. To verify the signature, check whetherm = (σe mod n).

Another source of asymmetry to create digital signatures comes from time. This idea is ex-ploited by the Tesla [62] protocol. The construction assumes nodes are loosely synchronized;using this, nodes can generate signatures that can be checked at a future time:

• Assume there is a one way function h. This means that given a value s, computing h(s) iseasy, but given h(s) it is not possible for an adversary to find s.

• Invoke the function h a number of times t using the output of the previous invocation asinput for the next one, the value for the first invocation will be s, this structure is known asa chain. In mathematical terms, this is: h0 = h(s), hi = h(hi−1). Publish ht as the publickey.

• During the first period of time, compute a MAC involving the message m, using ht−1 as akey to a fast symmetric cryptographic function (e.g. HMAC [63] explained in section 2.4.3).Finally, publish the result of the MAC along with ht. In practical terms, this means that we


h1 = h(h2||h3)

h2 = h(h4||h5)

h4 = h(h(m1)||h(m2))

∗h(m1) h(m2)

∗h5

.

∗h3 = h(h6||h7)

h6

.h7

h(m7) h(m8)

Figure 2.2: Example of a signature for m2 using a Merkle tree, appended nodes are marked with(∗).

provide an authentication function based on a key ht−1 along with a commitment to the keyrepresented by ht = h(ht−1).

• Nodes cannot check a signature for the current period, but once the period expires, the secretkey ht−1 is revealed. Nodes can verify this is the right key by computing h(ht−1). Securitycomes from the fact that it is computationally impossible for an adversary to simulate thechain, unless they actually know s or they can invert the one way function h.

Since no expensive number theory functions are used, this scheme is very efficient, but incurs inan initial delay. A real time authentication function can be achieved if the source can performinitial packet buffering [64].

A similar scheme known as the the Wong and Lam scheme [65] provides fast stream authen-tication but replaces the chain for a binary tree. In this scheme, the root of the tree is deliveredreliably. To verify a particular message mi, h(mi) and the siblings of its ancestors are appendedto the message (h is instantiated using cryptographic hash function such as SHA [66]). Fig. 2.2contains an example of signature for messagem2, here the nodes of the tree appended to the mes-sage have an asterisk (∗). To verify the signature, the verifier simply recomputes the tree usingthe given information. If the root matches the digitally signed data, received at a previous stage orfrom a trustworthy entity, the node can conclude the message is authentic.

2.4.2 Blom’s Scheme

Blom’s scheme [56] allows every user in a network to share a secret with any other user in anefficient way. LetD be a secret random c× c symmetric matrix and G be a public c×n generatormatrix of a Maximum Distance Separable (MDS) code, this is a code that meets the Singletonbound: for a (n, k, d)q then k = n − d + 1, Reed-Solomon codes [10] have this property. Allelements from both matrices and operations are carried in the field Fq. The set of secrets of thesystem is given by A = (DG)TG, where (DG)T denotes the transpose of (DG). The followingderivation shows that A is symmetric:

A = (DG)TG = GTDTG = GT (DG) = AT . (2.1)

The shared secret between users i and j is Aij = Aji. To compute this value, the i-th user receivesthe i-th row from the matrixKi = (DG)T . To get the shared secret Aij , i computesKiG∗j , where


G∗j is the j−th column of G, similarly j computes Aji = KjG∗i. If x ≤ c attackers collude, theyget no information about secrets that are not in the collusion.

2.4.3 HMAC

HMAC [63] is a cryptographic MAC based on a hash function h and a random key k (|| denotesthe concatenation operator). Given the information m to be authenticated

HMAC(k,m) = h((k ⊕ opad)||h((k ⊕ ipad)||m)) (2.2)

where opad=0x36|| . . . ||0x36 and ipad=0x5c|| . . . ||0x5c are fixed values padded to match thekey length. If a node wishes to verify that a node in possession of k created a message, it mustrecompute the function from the received message and then check if the output matches what wasreceived. Proofs of security for HMAC are found in [67].

2.4.4 Cryptographic Accumulators

Cryptographic accumulators were first proposed in [48]. As explained in [68]: an accumulatorscheme is an algorithm to combine a large set of values into one, short accumulator, such thatthere is a short witness that a given value was indeed incorporated into the accumulator. In thissection, we will introduce two accumulator constructions, one based on a Merkle Tree [37] andanother one based on a Bloom Filter [69]. The relevance to our construction is that in one of themthe witness is the message itself, whereas in the other additional information must be provided.Our proposal can be instantiated with any accumulator as long as it is hard for an adversary toforge membership for fake elements.

The first of the accumulators is based on a Merkle Tree which is a full binary tree where theleaves are the hashes of the elements that are included in the accumulator. This is the idea theWong and Lam scheme [65] described in Section . Internal nodes of the tree are computed as thehash of the concatenation of the children of that node. The root of the tree is authenticated andtransmitted to nodes in the network. The hash function h used for this accumulator must be hardto invert (preimage resistance). To prove an element is in the accumulator, the element, along withthe siblings of its path to the root are transmitted. Using these values it is possible to reconstructthe path all the way to the root of the tree. Security properties of the system follow from thepreimage resistance of the hash function, since it is unfeasible for an attacker to create a valid pathto the root, without inverting the hash function. The process is illustrated in Fig. 2.2 where it isshown what information must be sent to prove the membership of m2. for this case m2 and thesiblings of the ancestors of h(m2) can be used to reconstruct the path to the root.

An approach where only the message is necessary to prove membership, was presented in [70]by Nyberg. To add an element y to the accumulator, one computes y = h(y), here it is assumedthat y is rd bits in size. Next, y is treated as r different d-bit numbers (y1, . . . , yr), for each yia single bit bi is output, where bi = 1 if yi = 0 and 0 otherwise. The value for the accumulatoris the binary AND operation of the respective bit vectors computed from the elements. To verifythat an element y′ is in the accumulator, it is checked that each b′i corresponding to y′ is not 0in a position where the accumulator is 1. Unlike the Merkle-tree-based approach which needs an


m2 m1 m3 m4 c2 c1

Coefficients! Payload!

Packet =! H = cryptographic hash

function. E.g. SHA3!

H(Data) =! 10010011

This is a number!

10 01 00 11 Cut it as r numbers

of size d bits! r=4!

d=2!

1 1 0 1

If the d-bit number is 0 at all bits,

replace it by 0; otherwise 1 !

The accumulator is the

binary AND of this

result with the result

of the other packets

to be added!

Figure 2.3: Example addition of data to Nyberg’s accumulator [70] on a sample network codingpacket that is divided in coefficients and payload.

additional witness, only the message is needed. An example of this accumulator is presented inFig. 2.3 where r = 4, d = 2 and H=h.

Cryptographic accumulators in network protocols, have been used in a construction calledDis-tillation Codes [71], which was used for multicast authentication. In that construction, informationto be transmitted is cryptographically authenticated and then encoded using an erasure code, theoutput of the erasure code is added to the accumulator. Then, the encoded pieces are sent alongwith a proof of membership to a given accumulator. Receivers can associate encoded pieces todifferent original packets, since it is impossible for an adversary to create an encoded piece thatpasses the accumulator test. Once enough encoded pieces have been received from a particularaccumulator, the decoding procedure is invoked. If the decoded information produces informationthat can be authenticated, the data is accepted or discarded otherwise. Our constructions fromSection 2.6 can be seen as an extension of this construction, since the original did not considerscenarios where encoded pieces could be further encoded by intermediate nodes.

2.4.5 Related Work on Pollution Prevention for XOR Network Coding

In [72] a MAC-Based system was proposed. The idea of their construction is to use random el-ements of the packet’s payload to create a single MAC. The process is shown in Fig. 2.4, wherethe payload is represented by d1, d2, d3, d4 and the computed MACs by m1,m2, the arrows indi-cate what positions of the payload were used to create a particular MAC. In particular, m1 wasformed using d1, d2 and d5. The actual computation involves XORing the involved positions andthen XORing a random value generated by a secret key. The process of combining two packets,involves XORing the payloads of the two packets together and appending the MACs of both pack-ets to the resulting one. The process is shown in Fig. 2.5, where the payloads are representedby vectors d1, d2. As a result of this procedure, the packets increase in length depending on howmany of them have been combined so far. To verify a packet, the generation routine is repeatedand compared to the result of the XOR of the selected positions in the received packet.

In [31], a homomorphic MAC construction that can be used for XOR network coding wasproposed. Its construction is as follows: create a network coding matrix, each element inside the


d1 d2 d3 d4 d5 m1 m2

Figure 2.4: MAC computation from [72]. Random positions of the payload d1, d2, d3, d4 areselected for a particular MAC m1,m2, selected positions are XORed together and then XORedwith a value generated using a secret key.

d1 m1 m2

d1′ m′

1 m′2

d1 ⊕ d1′ m1 m2 m′

1 m′2

Figure 2.5: Combination step from [72], where two packets are combined into a single one. Theoriginal payloads d1 and d1

′are XORed, whereas all the MACs from the original packets are

appended to the resulting one.

matrix will be an element of F2, this matrix has the form [ID], where I is the identity matrix andD is the original message to be transmitted in matrix form. A random vector from v with elementsin F2 is generated using a private key and [ID] · v is computed, a second key is used to generate asecond vector w ∈ F2. A homomorphic MAC for a vector formed as a linear combination of therows of [ID] is given by

∑

r∈Rows([ID])

vr ⊕wr (2.3)

The output of the MAC is an element of F2. This MAC can be used on larger finite fields and theprobability of an adversary guessing the right value for a given packet is given by 1/|F|where |F| isthe size of the finite field. Since we are interested in vectors with elements in F2, an adversary canguess the right value with probability 1/2, for this reason several MACs need to be computed toachieve enough security. Packets can be XORed together and nodes in possession of the keys canverify it is not polluted. If internal adversaries are considered, even more keys are needed to protectthe system from keys known to them. As in any MAC-based system, the key assignment strategyis important to create a trade-off between security and information overhead. Unlike the previousscheme, packets do not increase their length as more XOR operations are performed. In [55], theauthors proposed a similar system. Like the previous construction, a random vector is XORed tothe information to be transmitted. The difference is that by using loose time synchronization, thesecret value used to create one MAC is sent after the message has been transmitted. This secretvalue is authenticated using a fast mechanism. Since the secret is unknown when the packets weretransmitted, receivers can be sure the message is authentic.


h1,i

h2,i

.

h(Ai,1||i, 1) .

h5,i

.

h3,i

h6,i

..

. h(Ai,c||i, c)

Figure 2.6: Computing the hash of the i-th row in A (Ai∗).

2.5 Proposal to Identify Misbehaving Nodes

As pointed out in Section 2.2, our proposal relies on an existing pollution prevention scheme,which we will denote as a Source Message Authentication Codes (SMAC). MACs introduced byour proposal will be denoted as Relay MACs (RMACs). The key idea of the protocol, is that itguarantees authentication for all messages using a fast function, before the slow SMAC verifyingroutine is applied.

2.5.1 Initialization

The sources initialize every node in the network and themselves, with two sets of secret values Si

and Ri. The purpose of Si is to give the node the ability to authenticate information sent by thesources; this information will be modified using network coding operations. Ri is given accordingto Blom’s scheme, its purpose is to generate shared keys kij to authenticate messages amongneighbors. Recall from Section 2.4.2 that these values correspond to the i−th row of the matrix(DG)T ; in addition, the identifier i is also provided to the node. A digital signature of all the keysin the system is given to each node, along with information to produce a proof that one key wasassigned by the source to a particular node. We will now explain how these are generated.

To avoid the use of digital signatures based on number theory at the node, we will use the treeconstruction to sign the whole square matrix of mutual secrets A (section 2.4.4). The idea is toembed A in the tree, as a set of row vectors appended one after another.

Recall from Section 2.4.2 that the secrets given to node i, are represented by the i−th row ofmatrix A, that will be denoted by Ai∗. The first step taken by the source is to produce a hash ofeach row in A independently, by using each coordinate of Ai∗ as a separate message for a Merkletree, whose root for each row will be denoted by h1,i. Fig. 2.6 shows this procedure; the reasonwe concatenate the actual coordinates of the secret to the invocation of the hash function, is toguarantee the secret was in that given position in matrix A.

Next, create a vector v = (h1,1, h1,2, . . . , h1,c−1, h1,c) where each component is the root ofthe tree computed from each row. Now we use this vector as an input to another tree whose rootwill be named σA. σA is the signature for the whole matrix A. In this computation, the additionalstring is not needed.

The only remaining thing to give the nodes, is information to prove other nodes, one secret wasassigned to them by the sources. To accomplish this, nodes receive, the siblings of their ancestorsin the Merkle tree created from v. The reasoning behind this, is that only node i is able to replicate

2.5. PROPOSAL TO IDENTIFY MISBEHAVING NODES 39

id r m SMACS RMACS

Figure 2.7: Packet structure

the path for any of its secrets until h1,i; then, by using the information from the siblings from hisancestors, any node can verify the result without contacting the source.

2.5.2 Transmission

When any node i (including the source) wants to transmit a message, it creates a packet as de-scribed in Fig. 2.7, the description of each field is as follows:

• id: Id of the the last relay that processed the message; for this case i.

• r: Increasing value used to derive a different key for computing RMACs.

• m: Data to be sent.

• SMACs: A suitable pollution prevention scheme.

• RMACs: One RMAC for each one of its neighbors j; first, a master key is derived for eachneighbor κij = HMAC(salt, kij), where salt is a random public system parameter; kij isderived using Blom’s scheme. The key used to generate an RMAC for node j is: k′ij =

HMAC(κij , r). Next, let m′ = (id, r,m,SMACS); the output of one RMAC function is(idj ||HMAC(k′ij ,m

′||idj)).

2.5.3 Relay Processing

When node j receives a packet, it verifies the RMAC intended for it. If the RMAC is not authentic,the packet is discarded immediately; otherwise, the node stores the packet until the SMAC in thepacket can be verified (buffering is common in Tesla-based protocols [55]).

If the verification for SMACs is successful, the packet is coded with other packets by gener-ating a linear combination of them; otherwise, the node increases a “bad events counter” for thesender and calls the decision routine (section 2.5.4); this counter is set to 0 periodically. If thedecision routine returns “true”, packets from sender i are not relayed anymore. To inform othernodes of the discovery of a misbehaving node i, entry Aji = Aij is sent to the neighborhood,along with information from the internal part of the Merkle tree only known to node i. We wantto point out that the whole internal tree needs to be computed only once, so it can be queried laterto accelerate the procedure. The total amount of information stored is twice the number of nodesin the network times the size of the output of the hash function (e.g. for 105 nodes and a 256-bithash function, memory used is 6.1MB).

As it was stated in the initialization stage, nodes complement their part of the tree with themessage received, if the result of the tree computation equals σA, the secret is considered legiti-mate. Note that the number of hash invocations is 2 log2(n), where n is the number of nodes inthe network.


Consider now node l who receives the new secret and who still considers i as honest; then,l uses that key to check the RMAC authenticity of packets coming from i, whose SMACs havenot been authenticated yet. Every time an RMAC is verified successfully and the correspondingSMAC is invalid, we increment the counter for “Bad events” proportional to the number of validRMACs we can check. If the number of bad events from node i exceeds threshold x, l labels i ascompromised and reveals Ail = Ali to his neighbors.

2.5.4 Decision Routine

This routine is only called when either: 1. A node presents a valid RMAC for another node; 2. Themessage authenticated by the SMAC is not authentic. Assuming there are c (or less) compromisednodes, an attacker can cause the first event by either, forging the SMACs for one node and lettingan honest node forward the packet (attack I); or forging the RMAC himself (attack II).

In attack I, we assume the attacker can forge a valid SMAC with probability 1/p; thus, theprobability in succeeding in this attack v times is:

(1

p

)v

· 1 (2.4)

here, the first term is the probability of an attacker forging the SMAC v times. The second term, isthe probability of the second event happening; the value is 1 because an honest node will alwaysforward packets that pass the verification routine for SMACs.

In attack II, the first event occurs with probability 1/|RMAC|; hence the probability of the firstevent happening v times is given by 1/|RMAC|v . If both events happen v times, the probabilityof labeling an honest node as compromised is given by:

1

|RMAC|v·(

1−1

p

)v

(2.5)

which is the probability of submitting a valid RMAC, times the probability of the validated mes-sage to be wrong by chance. Both events are considered independent because the RMAC andSMAC routines are completely unrelated due to their construction. The system administrator candetermine a threshold that is reasonable for false positives depending on the system parametersand equations (2.4) and (2.5).

2.5.5 Security of the Proposal

Our scheme relies on two protocols, the first of them is the authentication protocol, which relieson secret keys used to generate a random looking one-time key using HMAC.

Using this one-time key, the node invokes HMAC again to produce the definitive output of theRMAC function. For an attacker without the key, an RMAC looks random; thus, the probabilityof an attacker guessing a valid message-RMAC pair is (1/|RMAC|). When we apply this to the

2.6. PROPOSAL TO DETECT POLLUTION IN XOR NETWORK CODING 41

security game from section 2.2, the result is:(

1−1

ρ

)(1

ρ

)

−(1

ρ

)(

1−1

ρ

)

= 0 (2.6)

where ρ = |RMAC|; satisfying our security requirement.To prove the protocol construction guarantees authentication in our scenario, consider the

following: information needed to create a shared key is known by 3 entities, namely i, j (thetwo nodes who want to interact in the protocol) and the sources. In our scenario the sources areincorruptible, hence node i knows the MAC was produced by j, because no other node knows thekey.

The other situation that needs to be analyzed is when keys are revealed; we consider the tworelevant cases:

• A collusion of attackers I reveals the secrets AIj where j is honest. This is equivalent tothe attackers banning themselves from the network.

• A collusion of attackers I reveals the secret AIj to falsely incriminate j with node l, bothj, l are honest. Any RMAC computation by l, using secrets from AIj as input, is used onlyafter a message has been authenticated using Ajl; since Ajl is unknown to I , this attack isno better than faking an RMAC for key Ajl as long as |I| is within the collusion bound. Inpractice, special care must be taken to set threshold x to manage this attack properly.

2.6 Proposal to Detect Pollution in XOR Network Coding

As pointed out in Section 2.3, our idea consists in creating a trade-off between the size of the list ofauthenticated packets and performing decoding operations. In more detail, the scenario depictedin Fig. 2.1, shows a classical butterfly topology for a generation of size g = 4. The binaryvalues transmitted among the different nodes represent what packets were XORed to produce thegiven packets. To keep the example simple, the packets contain no payload. If we apply thestraightforward approach of signing all the possible combinations for a generation, it would benecessary to sign 24 = 16 packets. However, the list of authenticated packets L which is known toall nodes, only contains 6 elements. In the figure, node 3 performs the XOR of the two incomingpackets (1, 0, 0, 0), (0, 0, 1, 1) ∈ L, to compute (1, 0, 1, 1) and transmit it to nodes 4, 5. Given thatpacket (1, 0, 1, 1) ∈ L, nodes 4, 5 do not know whether it is polluted or not. Fortunately, node 4received (1, 0, 0, 0) from node 1 and node 5 received (0, 0, 1, 1) from node 2. If nodes 4, 5 applytheir received authenticated packets to the newly received one, the result is a packet in L.

This approach is known in the literature as “signature amortization” [73], because it amortizesthe cost of verifying a single signature among all the packets contained in the list. At first glance,the system might seem inefficient, given that we need to sign 2g packets. However, this is notnecessarily the case as shown in Fig. 2.1, where decoding steps are performed to reduce the sizeof the list of authenticated packets. The actual implementation of our idea does not rely on a listbut on the use of a cryptographic accumulator, which is a compact way to verify the packet is onthe list without transmitting the list.


1

3

2

4

5

6

(1,0,0,0)

(1,0,0,0)

(0,0,1,1)

(0,0,1,1)

(1, 0, 1, 1)

Figure 2.8: An example without instant decodability. Node 4 cannot forward packet (1, 0, 1, 1)because it is not in the authenticated list of packets. The list of authenticated packets is the sameas Fig. 2.1

In cases where it is not possible to reduce the packet to one that belongs to the list, such asthe topology depicted in Fig. 2.8, we propose a method to create the list and also that makes thebest use of the packets available in the list. We prove that when our method is used to repre-sent any packet as a combination of packets that can be authenticated, the overhead in terms ofretransmissions is never larger than the logarithm of the generation size.

In this section, we describe 3 variants which depend on the size of the generation involved, thenumber of packets that are XORed together (degree) and the coding strategy used by nodes in thenetwork.

2.6.1 Exhaustive Inclusion Strategy

For a XOR network coding network, given a generation of size g, there are 2g possible packetcombinations. For small values of g, it is possible for a source to compute all the possible XORcombinations among the packets and add them to a cryptographic accumulator.

Initialization: Add all the possible packet combinations to the cryptographic accumulator.

Encoding: Encode packets according to some scheme. The details of the coding scheme arenot relevant this variant of the system since all possible combinations are included in theaccumulator.

Verification: Verify every incoming packet is in the cryptographic accumulator; discard other-wise.

To make the computation more efficient, XORs of the different combinations can be computedusing the binary Gray Code. In this way, it is only necessary to compute the XOR of one packet perround, before adding the value to the authenticator. Authenticators needed for this constructionmust not have a proof of membership besides the message itself, otherwise packets cannot beauthenticated. Nyberg’s authenticator explained in Section 2.4.4 has this property.

2.6.2 Inclusion Strategy for Immediately Decodable Network Coding

The drawbacks of the previous construction are computational overhead at the source and a con-straint on the generation size. These drawbacks can be overcome by adding a constraint on the


N1

N3N2 N4 N5

Initital StateN1: A,B,C,DN2: A N3: BN4: C N5: D

Figure 2.9: Sample network topology with nodes N1, N2, N3, N4, N5 and packets A,B,C,D.The state shows what packets have been received by each node.

coding strategy of the nodes. A popular strategy for XOR network coding called Instantly De-codable Network Coding (IDNC) [3], guarantees that packets are decodable at the next hop (inFig. 2.1 this is the case). This guarantees that after one hop, nodes can use the accumulator toefficiently verify the decoded packet was transmitted by the source, for this reason only the orig-inal packets must be included in the accumulator. The information to select the right packets toachieve this property comes from feedback sent by the nodes; using this information, transmitterscan select a set of packets to code that satisfies the needs of the receivers.

Initialization: Add packets of degree 1 to the cryptographic accumulator.

Encoding: Encode maximizing the nodes that can decode packets immediately. This can be doneby using an algorithm such as the BronKerbosch algorithm [60] to find cliques in the graphrepresenting the coding opportunities, or specialized algorithms such as the one presentedby Le et al. in [28].

Verification: Reduce the degree of the incoming packet and verify it is in the accumulator; dis-card otherwise. Given the IDNC coding strategy, this process is performed by reducingthe degree of the incoming packet using authenticated degree-1 packets. Then, verify theresulting packet is in the accumulator.

This construction is similar to the exhaustive inclusion one; the difference is that only pack-ets of degree one are present in the accumulator. For this reason, it is necessary that the codingstrategy limits itself to transmit packets that can be immediately decoded by the receivers. Themain advantage of this construction compared to the previous one, is that the size of the generationcan be much larger. This also implies that computation at the source can be significantly smaller.Regarding the selection of the cryptographic authenticator, in this case it is possible to use au-thenticators with a separate proof of membership, such as the Merkle Tree construction. This ispossible since the accumulator only contains packets of degree one.

2.6.3 General Inclusion Strategy

Despite its efficiency and popularity, IDNC does not necessarily offer optimal throughput. Con-sider the scenario depicted in Fig. 2.9 and the sequence of packets transmitted by N1 shown onTable 2.1. On the left table, strickenout packets are those that have been received by nodes, butthat cannot be decoded immediately; since IDNC does not consider buffering these packets, theyare discarded. This is the case for packet A⊕C which cannot be decoded by N3 and N5. In thegeneral case, non-decodable packets can be used later to improve throughput, as it can be seen inthe right part of Table 2.1, where only 3 packets are needed as opposed to 4 in the IDNC scenario.

The steps for this scenario can be summarized as follows:


Table 2.1: Difference between immediate decoding and the general case from decoding perspec-tive

Immediately Decodable CaseN1 sends A⊕ CN1: A,B,C,DN2, N4 : A,CN3 : B,A⊕ CN5 : D,A⊕ CN1 sends B ⊕DN1: A,B,C,D

N2, N4: A,C, B ⊕DN3, N5: B,D

N1 sends D ⊕ CN1: A,B,C,DN2, N4: A,C,DN3, N5: B,C,DN1 sends A⊕B

N1, N2, N3, N4, N5: A,B,C,D

General CaseN1 sends A⊕CN1: A,B,C,D

N2: A,C N3: B, A⊕ CN4: A,C N5: D, A⊕ C

N1 sends B ⊕DN1: A,B,C,D

N2, N4: A,C, B ⊕DN3, N5: B,D, A⊕ C

N1 sends D ⊕ CN1, N2, N3, N4, N5: A,B,C,D

Table 2.2: Possible combinations for a generation of size 4

Packet Combinations(0, 0, 0, 0) (0,0,0,1) (0,0,1,0) (0,0,1,1)(0,1,0,0) (0,1,0,1) (0,1,1,0) (0, 1, 1, 1)(1,0,0,0) (1,0,0,1) (1,0,1,0) (1, 0, 1, 1)(1,1,0,0) (1, 1, 0, 1) (1, 1, 1, 0) (1,1,1,1)

Initialization: Add combinations that are likely to occur in the accumulator.

Encoding: Encode maximizing throughput according to some strategy.

Verification: Verify every incoming packet according to Algorithm 1.

In this scenario, finding packet combinations that are likely to occur in the system is not aseasy as in the IDNC scenario, because they depend on the topology of the network, packet lossrate, interference, noise and other factors. Since we are concerned with generic strategies that canwork for any network, we suggest adding packets to the accumulator whose degree is a power oftwo. An example for a generation of size 4 is presented in Table 2.2 where a 1 in the i-th positionindicates that the i-th packet will be coded. Packets included in the accumulator are written inbold letters. The motivations for this strategy is that without knowledge of the current system, itguarantees that any packet outside the accumulator can be produced using a logarithmic numberof packets (in the degree of the packet) that can be authenticated. This is useful in cases whenthe best packet for a particular situation is not contained in the accumulator. In that case it isguaranteed that a particular packet can be replaced by a not too large number of packets that canbe authenticated. We formalize the previous claim in Theorem 1.

Theorem 1. Let p be a packet that must be sent to neighbor nodes n1, . . . , nη. Let P = {p1, . . . , pδ},be a set of packets such that

⊕δi=1 pi = p where the degre of all pi = 1. Let g be the size of a

network coding generation and A be an accumulator containing all packet combinations where


Algorithm 1 General Pollution Detection Algorithm1: function VERIFY(p, T, V, L, τ )2: T ← T ∪ {p}3: if |T | > τ then4: Remove a random element from T5: end if6: repeat7: changed← FALSE8: for each t ∈ copy(T ) do9: Remove t from T10: B ← Reduce(V,t,L)11: if B is empty then12: Add t to T13: end if14: for each b ∈ B do15: if b ∈ Accumulator then16: Add b to V17: changed← TRUE18: else19: break20: end if21: end for22: end for23: until changed = FALSE24: end function

the degree of the combination is a power of two. Then, p can be transmitted using O(log (δ))

packets by performing combinations of elements in P .

Proof : Since all packets pi ∈ P have degree 1 and are different by the definition of a set; then,P ⊕ pi has degree δ − 1 and pi ⊕ pj = 2 for i = j. Now, select without replacement, c elementsfrom P , where c is the largest power of two which is less than or equal to δ and apply the XOR ofthe selected packets to p.

p′ ← p⊕ (p1 ⊕ . . . ⊕ pc) (2.7)

here, (p1 ⊕ . . .⊕ pc) is the packet to be transmitted.Since the degree of (p1 ⊕ . . .⊕ pc) equals c, because packets are different; then, the degree of

p′ is δ′ = δ − c. The process is iterated until the degree of the remaining packet is 0. Given thatin each iteration c ≥ δ/2, then 2δ′ ≤ δ. This shows that the degree of the original packet can bereduced by at least half on each iteration, hence the algorithm needs O(log (δ)) subsets of P toproduce the original packet p.

What remains is to show that the packets needed to represent p are in the accumulator. By thedefinition of accumulator A, packets whose degree is a power of two are included; therefore thepackets found by the algorithm belong to the accumulator.

The final step, involves checking that incoming packets are not polluted, this part is handledby Algorithm 1. Algorithm 1 first starts by adding the received packet p into the set of temporary


Algorithm 2 General Pollution Detection Algorithm1: function REDUCE(V, t, L)2: M ← RR|V |+1([vi ∈ V, t, li ∈ L]

′

)3: Z ← vectors li ∈ L that became [0] for the first time4: B← a basis for the linear span of Z5: return B6: end function

packets T which have not been verified for pollution. When the number of buffered packets in Texceeds the available memory, one packet is evicted randomly.

The next step involves reducing the degree of the packets in T , using elements from the setof already verified packets V . This could be done using Gaussian Elimination (GE) in F2; byreduction we do not necessarily mean decrease the actual Hamming weight of the coefficientsvector, but rather use it to find vectors belonging to the accumulator. What “Reduce” does, is toperform row reduction on the matrix [vi ∈ V, t, li ∈ L]

′ using the first |V | + 1 rows as pivotrows which we represent using RR|V |+1([vi ∈ V, t, li ∈ L]

′

), here “′” represents the transposeoperator. Then, it should find what members of li became the [0] vector, which means theseauthenticated packets can be created as a combination of the existing ones. Since this process iscomputed several times, some members of li will be [0] after they can be created for the first time;in that case, we only care about the first time this happens for each vector li. It is important tomention that just discovering a new packet, does not mean the packet provides new information;for a packet to actually provide new information, it must be linearly independent from the previousmembers of V . The previous insights are contained in Algorithm 2, where the matrix is reduced,then new vectors in the accumulator are selected and stored in Z , and finally a basis B of Z isselected as the newly decoded useful packets that can be tested in the accumulator.

2.6.4 Security Analysis

From a theoretical perspective, the three variants of our proposal guarantee that packets are nevercombined with other packets, until their memberships have been tested in the accumulator. For thisreason, the protocol is as secure as the accumulator used. From an implementation perspective,in the three variants, nodes must verify the membership of incoming packets. This could be usedby an attacker wishing to exhaust nodes’ resources, hence accumulators with low computationalcomplexity must be selected. In particular the accumulators presented in Section 2.4.4, wereselected for explanation purposes, because they are compuationally light. Despite this, Nyberg’saccumulator [70] must be used with utmost care in practice, since it has a high false positive rateif parameters are not selected correctly.

An additional security concern is present for the general inclusion strategy, given that packetsare evicted from the queue of unverified packets whenever the queue is full. An attacker that isable to inject a significant number of packets may cause eviction of valid packets. To deal withthis attack, an aditional reputation mechanism could be used to prioritize eviction of nodes whodo not provide enough valid packets. To prevent abuse of this measure, it should be enforced withan identification mechanism.

2.7. SIMULATION RESULTS 47

Table 2.3: Experiments in C

Algorithm Sign (µs) Verify (µs) Output (bits) SecurityRSA 3371 86 1024 280

RSACRT 1019 86 1024 280

RSACRTLE 1013 30 1024 280

DSA 450 790 320 280

Tesla 2 2 320 280

Proposal 32 bit 10 10 240 232

Proposal 80 bit 10 10 400 280

2.7 Simulation Results

2.7.1 Detection Routine

Tests were performed in an Intel Core 2 Quad 2.66 GHz Q6700 and Java JDK 1.6.0 21; all simu-lation values for times presented, are the average of 10000 runs.

In implementing the authentication protocol, we used HMAC as an RMAC function. Sincethe protocol involves creating an RMAC for each one of the neighbors, the overhead in this casewould be given by the number of neighbors times |RMAC| bytes in total, plus 2 bytes for the id ofthe neighbor. Assuming every node has an average of 5 neighbors and |RMAC| = 4, this wouldimply a total of 30 bytes in overhead. The time to perform the computation of the authenticationprotocol was 296.3 µs for a message of size 1500 bytes, assuming the keys had already beencomputed. The actual time for computing a shared secret during our experiments was around 1

ms, for a collusion bound of 1000 attackers, where arithmetic was performed modulo a primeclose to 2128.

To test how our method introduces additional delays, we implemented two fast, SMAC mech-anisms based on the product of random vectors in a finite field, one scheme was [55] and the otherwas [31]. The first of them computes the product of a random vector with each vector to be au-thenticated, then it delays the disclosure of the key used to generate the random vectors using theTesla protocol; the implementation of the protocol with an output of 1 byte, took approximately20 µs including the generation for the Tesla signature and the random vector using a SHA-1 basedrandom number generator from the Java Crypto API. The second one is similar but encrypts theoutput of the random vector to generate a MAC, our times for a packet with 5 MACs were about300 µs.

We evaluated authentication times using C++; the environment was g++ 4.6, GNU Multi-Precision library 5.0.1 − 11 for big numbers, and the SHA-1 implementation from Crypto++5.6.1−5. The algorithms we measured were RSA, DSA, Tesla based authentication and our MAC-Based proposal; results are summarized in Table 2.3. Results show that number-theoretic digitalsignatures are slower than their symmetric version counterparts, as it was anticipated; however, intheir favor they provide verification for an unbounded number of nodes.

The last two symmetric schemes are significantly lighter in terms of computation and over-head. The Tesla solution as implemented by ourselves, appends the previous hash in every packet.On the other hand, our construction scales linearly in time with the number of neighbors, for


this case 5 neighbors were used. Even though our construction is tailored to be used in the localvicinity of a given node, it improves the Tesla based construction since packets can be forwardedimmediately with the same security guarantees. Tag pollution is not an issue since the tags areonly meant for one hop.

The amount of storage needed by our authentication protocol is as follows: 1. c numbersas part of the matrix to generate the secret keys; 2. 160 · log2 c bits hashes for the path of thetree needed. For our experiments with 216 nodes, the total amount of information stored by thenode was 16320 bytes, assuming a collusion bound of 1000 nodes. This contrasts with only oneoutput of a hash function in [55] (160) bits; despite this our proposal is reasonable within thetechnical specifications of contemporary devices. Our system has similar memory requirementswhen compared to MAC-based systems used for pollution detection.

2.7.2 XOR Pollution Prevention

Simulations were performed on a Windows 8 PC, with 16GB in main memory and an Intel Corei7− 3770 CPU running at 3.4 GHz with an 8MB cache. The implementation of the cryptographicprimitives was made using the Java JDK 1.7 update 21. Unless otherwise stated, the executiontimes presented in this section, are the average of running the simulation 1000 times.

Exhaustive inclusion strategy

We performed several simulations to measure the efficiency of the proposal, the first of them isshown in Fig. 2.4. The table shows execution times needed to compute all the possible packetcombinations of a given generation as well as the variants that only add packets of degree 2k;for the columns using hash functions, a Nyberg accumulator was computed. To generate enoughcryptographic material the hash function was invoked several times, taking as input the previousoutput. The simulation wanted to verify the overhead induced by the selection of the hash functionin our proposal; the hash functions that were tested were MD5 [74] and SHA-1 [66] due to theiravailability in the Java JDK, however MD5 must not be used for a real implementation since itis no longer secure. The final column shows the execution times of just creating all the XORcombinations for the packets in the generation, without creating the accumulator.

As it can be seen from table results, the selection for the hash function greatly impacts theability of the source to create packets. This implementation used a single thread and a Grey Codeapproach to XOR a single packet per hash function invocation when all possible combinationswere added to the accumulator. A performance improvement is seen for the 2k variants of thealgorithms, since less number of combinations are computed, time is reduced, the cost that is payedin this case is that not all possible packets can be authenticated immediately, which influencesthe coding strategy of nodes. Even though the procedure could be performed in parallel, theexponential growth in the number of combinations confines this proposal to small generation sizes.This proposal is suited for information that is known in advance, such as software updates; forinformation generated in real time, it is possible to use it depending on the throughput needed. Forinstance, the SHA-2k implementation shown on the fifth column of Table 2.4, could be used totransmit at approximately 232.5 KBps, a parallel implemenation could scale this rate linearly andthe selection of a faster hash function can increase throughput as well to match application needs.


Table 2.4: Execution times in ms for creation using the exhaustive inclusion with 1500 byte packets

Gen. size MD5 MD5-2k SHA SHA-2k No Hash4 0.18 0.10 0.15 0.10 ms 0.028 1.14 0.47 1.69 0.70 ms 0.212 18.13 4.76 26.59 7.05 ms 3.0816 285.95 69.68 425.17 100.78 ms 49.16

Table 2.5: Execution times for creation using the IDNC variant with 1500 byte packets

Generation size MD5 (µs) SHA (µs)8 36.36 50.1416 59.47 93.5832 119.29 187.5164 238.96 382.15

Adding only packets whose degree is a power of 2 to the accumulator increases the numberof packets that must be transmitted. For the case of a generation of size 16 on average 2.03

packets are needed in the 2k version, compared to the optimal case where all packets are includedin the accumulator. This value was found by exhaustively checking all the possible coefficientcombinations for each generation; the value for a generation of size 32 was found to be 2.48.This value is consistent with Theorem 1 which gives an upper bound for a single packet. To putthis results into context, assume that in a network not using network coding, in order to satisfy theneeds of a group of nodes, n packets must be retransmitted. Let us say that by using XORNetworkCoding, this number can be reduced to r < n packets. What the previous result tells us, is thatfor a system using the 2k strategy for g = 16, in average the transmitter will send 2.03 · r packetsthat can be immediately authenticated to satisfy the needs of the receivers. Given the bound fromTheorem 1, the transmitter will never transmit more than 4 · r packets in that scenario.

IDNC

The next simulation involved the IDNC scenario, results are summarized in Table 2.5. Sincethis variant involves no XOR operations, the “No hash” column is not included. This variant issignificantly faster than the small-generation one, since only one hash invocation is needed perpacket. Results show that our implementation is in the order of µs instead of ms as in the previousscenario. For this reason, this construction can be used with real time content, for which IDNC isa good technique to improve throughput. For this variant a Merkle Tree accumulator was used.

General inclusion strategy

For the general case, testing whether an unverified packet generates a packet in the accumulatorfor a generation of size 16, takes approximately 47 µs in our Gaussian Elimination routine usingbitwise, for this reason the buffer of temporary packets must be kept small. For this experimentthe number of possible packets in the accumulator was set to 14827 since that is the number ofcombinations added when only packets with degrees that are a power of two are added. To reducethe processing time, it is recommended that a sender willing to use packets not in the accumulator,


Table 2.6: Execution times for creation and verification of existing schemes with 1500 byte packets

Scheme Generation (µs) Verification (µs)Agrawal et al. [31] 164 184Yu et al. [72] 216 209

should inform receivers of what packets in the accumulator are expected to be decoded. Thisreduces the complexity of the reduction function.

Comparison against existing schemes

Next, we compared creation times for two fast MACs that can be used for Network Coding, resultsare summarized in Table 2.6. Execution times for these constructions are in the order of µs aswell; the results show the execution times for the creation of exactly 1 MAC and verification ofthat MAC for a generation of 16 packets. Yu et al.’s construction from [72] changes its executiontime, depending on the number of packets that have been coded, verification time for this schemeconsider an uncoded packet, which is its fastest case. MAC generation times are slower than ourIDNC proposal; however, they are significantly faster than our proposal for the small generationvariant. Regarding security, in the case of Agrawal et al.’s [31] construction, many MACs mustbe computed since the output of each MAC is only one bit long, which reduces performance forboth generation and verification in real scenarios, where several keys must be used. Yu et al.’sconstruction [72] can produce individual MACs with larger security, but still several of them areneeded, since a single MAC does not protect all the positions in the payload. Both sytems alsosuffer from drawbacks associated to their MAC nature, such as key distribution and the overheadvs collussion resistance trade-off among others.

In terms of verification times, for all our variants they were in the order of 5 µs for MD5 and7 µs for SHA. This results apply to a network of any size with any number of attackers; this resultdoes not include the time needed to authenticate the packet defining the accumulator.

Regarding schemes using number theoretic functions [30, 33, 75], they were not considered inour simulation, since their performance is significantly slower than the MAC-based ones that wereanalyzed.

2.8 Conclusions

In Section 2.5, we presented a computationally light mechanism to isolate malicious nodes in anetwork. The mechanism works in the presence of a great number of attackers. Unlike otherproposals in the literature, we do not rely neither on being able to reach a central authority afterdeployment, nor on a particular topology. This holds as long as the underlying pollution detectionroutine is secure.

As an additional contribution towards makings network coding resisilient to pollution attacks,in Section 2.6 we presented a proposal with 3 variants to prevent pollution in XOR NetworkCoding. Unlike existing proposals, our construction does not rely on homomorphic functionsand can be instantiated using standard primitives in a practical scenario. On the three variants,

2.8. CONCLUSIONS 51

verification is in the order or µs due to the use of fast cryptographic accumulators. The limitationsof the variants are related to the size of generation or the kind of coding strategy they can handle.

The first variant is not concerned with the coding strategy used to forward packets, but canonly be used for small generations. The second one, does not have a limit on the generation size,but can can only be used in networks where Immediately Decodable Network Coding (IDNC) isused. For the IDNC scenario, during our tests, our proposal outperformed existing constructionson both generation and verification times. The final variant allows a more diverse coding strategy,but for realistic scenarios is confined to small generations. For the final variant, we developeda trade-off strategy with logarithmic retransmission overhead that reduces the amount of workneeded by the source.

The proposals for XOR Network Coding pollution prevention, present an interesting trade-offbetween the source generating the content and nodes forwarding it. This is useful for networkswhere relays and sinks have considerably less capabilities than the source.

Chapter 3

Regenerating Codes and Proofs of DataPossession

53

54 CHAPTER 3. REGENERATING CODES AND PROOFS OF DATA POSSESSION

3.1 Introduction

The use of outsourced data storage facilities has become increasingly important thanks to the avail-ability of cloud providers. These services promise their users, they will be able to focus on theircore business rather than in acquiring and managing the infrastructure needed to support their op-erations. Despite its advantages, this new model of data storage, also introduces some challengesin terms of guaranteeing users that their outsourced information is stored entirely and replicatedproperly. This chapter will present solutions to some of the challenges involving regeneratingcodes and proofs of data possession presented in Section 1.4.

One of the main benefits of outsourcing information is reducing the users’ burden of handlingwith long term information storage, which is not a trivial problem in practice. However, providersmust still deal with problems related to keeping the information available; therefore, informationmust be stored redundantly to avoid data loss in case of node failure. One way to deal with theproblem could be to use an erasure code [76], in which the file is stored at n servers such that anycombination of the pieces stored at k servers is able to reconstruct the original file. Even thoughthis is optimal in terms of space, there is one practical problem to apply this approach for cloudstorage. If one node fails, the entire file needs to be reconstructed before the failed node can beregenerated. To solve this problem, in [9] the authors proposed Regenerating Codes (RCs) whichare able to regenerate one node without reconstructing the entire file.

Another problem faced by cloud storage systems is how to prove they are still storing theinformation. This problem is addressed by a primitive called “Proofs of Retrievability” (POR) [15]and “Proofs of Data Possession” (PDP) [14], which are proofs that can only be passed by a prover,if it has access to the file. Even though the goal of both primitives is the same PORs provide anadditional guarantee that the file can be retrieved. We will refer to this primitive as POR duringthe rest of the chapter.

To address the redundancy requirement, several constructions have been presented in the lit-erature. For instance, in [77, 78] the authors propose using RCs guaranteeing authenticity; whilein [16] a system for possession of data was constructed. In terms of data possession, one stepfurther was taken in [22], where the authors considered how to apply PORs to files encoded usingregenerating codes. The limitation of their construction is that they do not consider errors thatmight be transmitted from one storage node to another when reconstructing a failed node; thiskind of error/attack is known as pollution. To solve it, they use an existing primitive to deal withthis problem. The challenge in designing PORs for encoded files is that traditional PORs are notmeant to be used with files that pass through an encoding process. In addition to the previousproblems, systems using encoded information must also guarantee that the original file can bereconstructed using any k servers. Traditional approaches towards this problem involve checkingthat all combinations of k servers can reconstruct the original data [77].

Contributions of this chapter come from several aspects, the first one is the PORs from Section3.3.1 and 3.3.2 which are different in nature from that of [22], easy to implement and can be usedfor pollution detection using a different key management scheme. The second is the proposals forkey management presented in Section 3.3.3 that can reduce processing times, when the numberof storage nodes in the network increases. They can also turn the system from POR to pollutiondetection routine with a different protocol interaction. The strategy itself is made to take advantage

3.1. INTRODUCTION 55

of the properties of network coding for pollution detection and the fact that the file owner is online.The construction can be extended to other linear codes as long as the encoding mechanism can beinferred from the incoming packets. Regarding decodability of the file from the encoded pieces,we present a mechanism to guarantee in polynomial time that any k nodes can reconstruct the file.The method involves using a Hilbert matrix to select the coefficients during the encoding process.We also present an application related to cloud music services, where users upload their localmusic libraries to the cloud. To make this procedure faster, cloud music services first compare if afile exists in their catalogue; if the file exists, it does not need to be transmitted over the network.This achieves deduplication on both network and storage. However, an adversary might simulatethe protocol interaction of the official client and the music service, to claim it has files it does nothave. As a result, the music cloud service might lose revenue, since it is giving away content forfree.

3.1.1 Problem Statement

There is a file M of length B data units, the data units in the file can be interpreted as membersof a finite field or vectors whose elements are on a finite field. When data units are interpretedas finite field elements, addition and multiplication operations will be those of the correspondingfinite field, whereas for vectors addition is the element-wise addition, the product will be the dotproduct of the two vectors. This is consistent with the matrix operations needed to encode thefiles.

The data units will be stored at n storage servers (which we will also refer as nodes), suchthat any k out of the n nodes can reconstruct the file. Also, any d ≥ k nodes must have enoughinformation to regenerate a failed node. Our objective is to provide routines to guarantee the filecan be retrieved at any given moment in an efficient way. By efficient way, we mean transferringsignificantly less information than the length ofM .

Our threat model considers nodes that might want to hide the accidental loss of data, but whenasked to provide a given piece of data, will reply to a request by either returning the stored datawhen it is available, or a random value otherwise. In addition, an external attacker might havecompromised a node inadvertently by polluting its contents which are then forwarded to a newnode during a regeneration operation.

We assume that the file owner can be contacted at all times and that it is able to store smallpieces of information for security purposes. In addition, the file owner can interact with the newnode during a regeneration operation.

The proposed scenario models a situation of a file owner that wishes to outsource file storageat several cloud providers for increased availability with low storage overhead to provide accessto them by (e.g. video portal). Given that there is liability involved when providers fail to storeinformation, they will do their best effort to comply to all requests from users. Nevertheless,they have financial incentives to hide problems in their infrastructure. We claim the assumptionof having the file owner interact with the servers, is consistent with an internet site outsourcingstorage while still being able to monitor the storage servers’ operation.

The rest of the chapter is structured as follows: in Section 3.2 we introduce an existing POR aswell as the existing construction of regenerating codes of our interest. In Section 3.3 we present our


POR construction for RCs and the coefficient selection strategy along with applications of PORsto cloud music services. in Section 3.4 we present the simulation results for our implementationof the proposal. Finally, in Section 3.5 we present the conclusions of this chapter.

3.2 Preliminaries

We will now present some of the required knowledge from PORs and regenerating codes neededfor our proposal.

3.2.1 Regenerating Codes

A Regenerating Code stores a file of length B into n different nodes, each one of them storing αsymbols, such that any k nodes can reconstruct the original file. When a node fails, a replacementnode can be created by connecting to d ≥ k nodes and downloading β ≤ α symbols from eachone of them. dβ is known as the repair-bandwidth. Compared to traditional erasure codes, regen-erating codes use a smaller repair-bandwidth to regenerate one node. Different constructions ofregenerating codes can focus in either minimizing the repair-bandwidth or storage α.

In the regenerating code literature there are two main approaches, namely functional regenera-tion and exact regeneration. In the first one, even though any k servers are still able to reconstructthe original file, the information of the regenerated node is different from the previous version ofthat node. In the latter, the regenerated node has the exact same information.

The original Regenerating Code is shown in Fig. 3.1. To regenerate a node, each cooperatingnode outputs linear combinations of its data units. In the figure these values are called a1, a3and a4; the replacement node then stores random linear combinations of the received values.The combinations used to produce these values, will be linearly independent with overwhelmingprobability when a large finite field is used. As it can be seen from the example, even thoughthe replacement node Cj has the same functionality as the old node N2, its contents are differentfrom the original. The reason we focus on this code, is because its method to regenerate a nodeis equivalent to adding a new node in the network. For this reason, the number of nodes in theis not fixed and can grow to match any desired level of redundancy, for either storage or contentdistribution.

3.2.2 Proofs of Retrievability

A proof of retrievability (POR), is a proof allowing a computer storing a piece of information toprove the information is still stored entirely [15]. The need for this kind of proof arises from thecloud computing scenario, where storage service providers, might have incentives to hide the lossof rarely accessed data, for purposes such as: cost efficiency and avoid liability in case of data loss.One simple way for a client to test the data is available, would be to download it entirely; however,this might be slow and costly due to the pay-on-demand business model of many providers.

In [15] an efficient homomorphic proof with the previous characteristics is presented. Considera fileM encoded into n data units (m1, . . . ,mn) where mi ∈ Zp and p is a large prime number;

3.2. PRELIMINARIES 57

Node Data unit 1 Data unit 2N1 m1 + 2m2 m3 +m4

N2 m1 +m3 m2 + 2m4

N3 m1 + 5m2 + 3m3 +m4 2m1 +m2 +m3 + 4m4

N4 5m1 + 3m2 +m3 + 2m4 3m1 + 2m2 + 5m3 + 3m4

(a) Initial State for Storage Nodes

Node Value DataN1 a1 m1 + 2m2 +m3 +m4

N3 a3 3m1 + 6m2 + 4m3 + 5m4

N4 a4 8m1 + 5m2 + 6m3 + 5m4

(b) Information Sent by N1, N3 and N4 to create a new node Cj

Coefficients N2’s regenerated data3a1 + 2a3 + a4 17m1 + 23m2 + 17m3 + 18m4

a1 + 2a3 + 3a4 31m1 + 29m2 + 27m3 + 26m4

(c) New node Cj used to regenerate N2

Figure 3.1: A Regenerating Code based on Network Coding [9]. There are 4 storage nodes storinglinear combinations of the original data unitsM = {m1, . . . ,m4}, any two nodes can reconstructM and any 3 can create a new storage node. To regenerate a given node, each node outputs alinear combination of its stored data units. In the example, node Cj , receives values a1, a3 anda4 and stores random combinations of them. Even though the regenerated data for Cj is differentfrom the original one, this does not alter the properties of the system; note also that M was notreconstructed to regenerate node Cj .

each data block mi is augmented with:

σi = fk(i) + γmi ∈ Zp (3.1)

here fk(i) is a Pseudo Random Function (PRF) using a secret value k and parameter i; γ is arandom secret value.

A client wishing to verify the file is stored at the server, chooses l indices at random in therange [1 . . . n] along with l random coefficients vi ∈ Zp to form the set of pairs Q = {(i, vi)}.The server then computes:

σ ←∑

(i,vi)∈Q

vi · σi µ←∑

(i,vi)∈Q

vi ·mi. (3.2)

Using the values computed by the server, the client can check if the following equality holds:

σ = γ · µ+∑

(i,vi)∈Q

vi · fk(i). (3.3)

To reduce the overhead, one authenticator can be used every s data units, which are calledsectors; the construction can be found in Section 3.2 of [15]. In case the sectors constructions isused, then the reply from the server consists of one µi per sector. The equivalent equation of (3.1)in the new system is given by:


σi = fk(i) +s∑

j=1

γjmij . (3.4)

The difference relies on the use of more random coefficients γ and a single call per sector tothe PRF. Putting this in context, (3.1) is the special case when sector size equals 1.

The protocol as explained is vulnerable to an attack where the attacker changes or permutes thetags of a given file. To prevent attacks of this kind, the original protocol considers authenticatingeach tag and its position within the file. We refer the reader to the original paper for all the details.

The previously explained POR has an interesting property, it can be used an unlimited numberof times, without reducing its security. In a framework presented in [49], proofs with this propertyare presented as unbounded-use schemes. We modified this construction to make it work for en-coded files in section 3.3.1. Even though this proposal is very attractive both from theoretical andpractical points of view, we focus our attention on the practical benefits of bounded-use schemes,since they provide lower communication overhead in practice.

To create secure bounded-use schemes, we can make use of the general framework presentedin Section 5.2 of [49]. During the description of the framework,M will denote the file in questionand ℓ the number of times the proof can be used. Despite authenticators are secure, they can onlybe used once, therefore, selecting ℓ depends on how often the tests will be performed and for howlong the file will be stored on the cloud.

• The user chooses a Pseudo Random Function (PRF) fk1(i), here k1 denotes a secret valueand i the input to the function. Then computes a set of challenges e(i) = fk1(i), where 1 ≤i ≤ ℓ and the answers to those challenges u(i) = Answer(M,e(i)). The actual descriptionof the answer is protocol dependent.

• The user chooses a random function fk2 and computes the authenticators σi = fk2(i, u(i));

then the authenticators are appended toM .

• The user picks a random one-way hash function h and sets ω = h(M). This value is notused in the verification procedure, but it is needed to prove that there is an algorithm thatcan reconstruct the file.

• The server stores (M,σ1, . . . ,σℓ), the client stores (k1, k2, h,ω, i), where i is a counterdenoting the next test to be performed.

• To perform a test, the user (verifier) sends the challenge information along with an index(e(i), i). The server (prover) sends the next σi authenticator, along with its answer to thechallenge (µ′,σi). The verifier approves if fk2(i, µ′) equals σi.

Note that k2 is unknown to the server, that is why given an authenticator σi, it does not provideenough information to check the computation is valid by itself. In the same way, the randomchallenge e(i) does not allow the prover to compute the answer in advance. We will use thisframework to propose a bounded-use construction for Functional Regenerating Codes (FRC) inSection 3.3.2.

3.2. PRELIMINARIES 59

3.2.3 Group Testing

Group Testing algorithms [42] can be classified in adaptive and non-adaptive. In adaptive algo-rithms, it is possible for the verifier to see the outcome of previous tests in order to design newones. In non-adaptive algorithms, the verifier must plan the tests in advance. The latter type isusually employed when tests require a long amount of time to be performed, in which case it ismore efficient to perform tests in advance.

The original group testing scheme wanted to minimize the number of tests t, by finding theoptimal pool size s, defective elements were assumed to appear with probability p. The proposedmodel was given by:

E[t] = n/s+ (n/s) · (1− (1− p)s) · s (3.5)

which is the number of groups of size s plus the number of groups that must be retested. Then,given n and p it is possible to find the optimal pool size s. In this scenario, depending on theoutput of the first iteration, it is possible to decide what elements would be retested. An exampleof non-adaptive group testing for 7 elements peforming 3 tests, that is able to identify at most 1defective element correctly is given by the following matrix, where each test is associated to acolumn. A number 1 in position (i, j) of the matrix indicates that the i−th element to be testedwill be included in the j−th test. Each xi is a binary value indicating whether a given element isdefective (1 for defective), this value is unknown before the tests are performed.

[

x1 x2 x3 x4 x5 x6 x7]

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0

0 1 0

1 1 0

0 0 1

1 0 1

0 1 1

1 1 1

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

(3.6)

Performing the tests, is equivalent to applying the matrix to the test. Let us say that the resultafter running the experiments is given by:

[

0 1 1]

(3.7)

where 1 means that one of the elements is defective. In that case the only possible vector of xivalues with at most one defective element, indicates that the 6th element is defective. This canbe inferred since the 6th element is the only element that is present in the tests whose output is1 and not present in the tests where the output is 0. The previous example shows the relationbetween non-adaptive group testing and coding theory, in fact the matrix was generated based onthe Hamming Code. The disadvantage from this approach is that the maximum value d must beknown in advance. Note that for this particular case, applying the three tests has the same resultsas the case when the 2nd and 4th elements are defective, therefore the cases are indistinguishablewhen d is exceeded.


Algorithm 3 BisectRequire: elems: set of elements with at least one defectiveEnsure: D: Set of defective elements1: D ← ∅

2: if elems.length() = 1 then3: D ← D ∪ elems return D4: end if5: f1 ← ApplyTest(elems.first half())6: if verify(f1) is TRUE then7: D ← Bisect(elems.second half()) ◃ The defective element is in the second half8: else9: D ← D ∪ Bisect(elems.first half())10: f2 ← ApplyTest(elems.second half())11: if verify(f2) is FALSE then12: D ← D∪ Bisect(elems.second half())13: end if14: end if15: return D

When d can take arbitrary values, it is necessary to use more general methods. In particularthe bisection method is very powerful and achieves the information theoretical bound. Algorithm3 shows this method, the idea behind it, is to apply a test over all the elements, if there are nodefective elements the algorithm stops; otherwise, the pool is divided into two smaller pools eachone with half the number elements. However, there is a subtle difference in the method to reducethe number of tests. If we have tested two elements mi and mj together and the result wasdefective, if we test element mi and is clean, then we automatically know that mj is defective,in total 2 tests need to be performed. However, if the second test was applied to mj , there is notenough information to know ifmi was defective; therefore, another test needs to be performed fora total of 3.

Other variants of group testing can be used not to find the exact number of defective elements,such as [79], where the goal is to estimate the number. We do not apply this approach in this work,since we need to find the exact blocks that are defective.

3.3 Proposal

In this section we present an unbounded-use POR and a bounded-use POR. For the unbounded-usecase, we present just the cryptographic primitive. For the bounded-use we present the interactionamong the participants. For the bounded-use proposal, the idea consists in a client c storing afile M using a regenerating code at several storage servers. When c wants to verify F is storedat S, it runs a POR on each one of the servers; this POR must guarantee the server containsa linear combination of the pieces that were entrusted to it, either by c or during regeneration.After this, c verifies the reconstruction is possible by any k servers; this is done by verifying thelinear independence of the coefficients stored by any k servers. Finally, a regeneration operationis simulated for each node. To avoid attacks where storage nodes provide polluted pieces of the

3.3. PROPOSAL 61

file (data units that cannot be produced as the linear combination of F ’s blocks), we present anefficient construction for that purpose.

3.3.1 Unbounded-Use POR for Encoded Files

We will model stored information as linear combinations of the parts of the original file M =

(m1, . . . ,mn) using random coefficients βi ∈ Zp. These coefficients are obtained from the defini-tion of the Regenerating Code. We will represent the linear combination of original pieces storedat a node by

X ←n∑

j=1

βjmj (3.8)

and its stored authenticator by

Θ←n∑

j=1

βjσj =n∑

j=1

βj(fk(j) + γmj) (3.9)

here the σj values are those computed by the original scheme from [15]. To verify an encoded file,c chooses a set Q = {(i, vi)} of random indices i (over all the possible combinations) and randomcoefficients vi ∈ Zp and asks the server to compute the following two values:

σ ←∑

(i,vi)∈Q

vi ·Θi =∑

(i,vi)∈Q

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βjσj

⎞

⎠

⎤

⎦ (3.10)

µ←∑

(i,vi)∈Q

vi ·Xi =∑

(i,vi)∈Q

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βjmj

⎞

⎠

⎤

⎦. (3.11)

The server passes the test if the following equality holds

σ = γµ+∑

(i,vi)

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βjfk(j)

⎞

⎠

⎤

⎦. (3.12)

The proof is sound as it can be seen in the following derivation. Replace (3.11) in (3.12) toobtain

σ = γ∑

(i,vi)∈Q

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βjmj

⎞

⎠

⎤

⎦

+∑

(i,vi)∈Q

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βjfk(j)

⎞

⎠

⎤

⎦ (3.13)


reorganizing the right side

σ =∑

(i,vi)∈Q

⎡

⎣vi ·

⎛

⎝

n∑

j=1

βj [fk(j) + γmj]

⎞

⎠

⎤

⎦. (3.14)

finally, by replacing (3.9) in (3.14), we get

σ =∑

(i,vi)∈Q

vi ·Θi. (3.15)

This matches (3.10), which is what we wanted to achieve.Regarding security, from the server’s perspective, the presented protocol is using the same as

the secure original version for non-encoded files. From the client’s perspective, in order to inferthe correct value for an encoded file piece, the right set of coefficients βj is applied during theprotocol execution. Because of this, the original construction can be seen as an instance wherean X value is stored at index i in the file, and its corresponding set of coefficients for the linearcombination are βj = 0 except when i = j where the value is 1.

It is important to mention that the previous security analysis, assumes that a given data unit ata server cannot be produced as the linear combination of other data units. Otherwise, by knowingthe coefficients of each data unit, it is possible to apply the proof to the other data units and obtainthe same result, due to the linear properties of the function. The effect this has on the final user, isthat even though it is possible to recover the data unit, the actual representation used by the servermight be different from what the user is expecting.

To verify a regeneration using this unbounded use scheme, the file owner performs the verifi-cation protocol with the new node several times. The first set of proofs is applied to the data thatwas received from the other nodes to verify that they are correct. If some proofs do not yield theright result, another node needs to be selected to complete the regeneration operation. If all theverifications are executed correctly, then the regenerated node executes the verification protocolwith the file owner, once it has computed the information that it will store. The file owner can thenperform proofs with the nodes that forwarded polluted information and take proper action.

3.3.2 Bounded-use POR for Encoded Files

To instantiate the framework from section 3.2.2, we propose the use of the dot product as the firstrandom function from the framework. Given its linear properties, the function is able to withstandthe encoding operations while still providing the desired security. This is our construction:

• Assume the code used to regenerate the fileM = (m1, . . . , mn) is the functional one fromFig. 3.1, where the network coding matrix has n rows. The vector notation is added forclarity, since the data units will be treated as vectors on this proposal.

• Using k1, generate a new key k′1 to create the challenge. This key will be used to generate ℓrandom vectors v(i) whose dimension is the number of columns of matrix M . The answerto the i-th challenge denoted by e(i) is µ(i) = v(i) · mj , where 1 ≤ j ≤ n. This correspondsto the dot product of the random vector with all the rows of the matrix.

3.3. PROPOSAL 63

• The next step involves computing a Message Authentication Code (MAC) using a differentrandom function fk2(i, j, µ

(i)). Since we need all σ values from a given test to be com-bined during a functional regeneration, then fk2(i, j, µ(i)) generates a random number r(i,j)

and outputs σi,j = r(i,j) + µ(i). For a given test, each σi has dimension, the number ofcomponents of basis vectors in the matrix.

• The server stores (M,σ1, . . . ,σℓ), the client stores (k1, k2, h,ω, i, mj · vj). The differencewith the framework definition is that the output of the dot products are also preserved, thisallows the client to recreate the computation using only the dot products. It is also possibleto store this at the servers, in which case they must be authenticated along with the index ofthe test.

• To perform a challenge, the client sends k1 to the server. The server computes its answerµ′ and sends it to the client along with the vector giving the information of the linear com-binations stored at that particular server and the corresponding σi. Using the coefficients,the client can know the correct value for the answer µ(i); the same coefficients are thenapplied to σ to see if the values match. It is important to note, that a mechanism must beimplemented to guarantee that the coefficients sent by the server are indeed those assignedto those blocks. Another option could be to write the coefficients in a predictable way, sothat they do not need to be transmitted.

Here is an example that shows some of the protocol properties. Consider a file of length 8

stripped across 5 servers where 4 of them are needed for reconstruction and random vectors v andr, as proposed in the protocol.

M =

⎡

⎢⎢⎢⎢⎢⎢⎣

1 0 0 0 f1 f2

0 1 0 0 f3 f4

0 0 1 0 f5 f6

0 0 0 1 f7 f8

1 1 0 0 f1 + f3 f2 + f4

⎤

⎥⎥⎥⎥⎥⎥⎦

v =

⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

5

7

9

3

2

4

⎤

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

M · v =

⎡

⎢⎢⎢⎢⎣

5f1 + 7f2 + 9

5f3 + 7f4 + 3

5f5 + 7f6 + 2

5f7 + 7f8 + 4

⎤

⎥⎥⎥⎥⎦

r =

⎡

⎢⎢⎢⎢⎣

45

48

97

80

⎤

⎥⎥⎥⎥⎦

(3.16)

The first thing to notice is that the overhead of this scheme depends on two aspects: the numberof storage nodes and the number of tests that can be performed. Note that the length of the file isnot important, just the number of nodes needed to recover; for this particular case, the client needsto store vector M · v to be able to predict the output of a functional repair. In the example onlyone test is shown, but as more tests are needed, each one of these vectors of dot products must bestored.


Assume a node i is regenerated and that the resulting linear combination stored at it, and itscorresponding σi are given by:

mi =3f1 + 2f3 + 5f5 + 4f7, 3f2 + 2f4+

5f6 + 4f8, 3, 2, 5, 4 (3.17)

σi =15f1 + 21f2 + 10f3 + 14f4 + 25f5 + 35f6 + 20f7+

28f8 + 1095. (3.18)

Note that this is different from the original set of nodes, but it is still a valid combination. When iis presented with v, it can compute:

mi · v =15f1 + 21f2 + 10f3 + 14f4 + 25f5+

35f6 + 20f7 + 28f8 + 59. (3.19)

This result can be replicated by the client who knowsM · v and the coefficients vector of node i,by computing the dot product between the coefficients vector of node i and matrixM · v:

⎡

⎢⎢⎢⎢⎣

3

2

5

4

⎤

⎥⎥⎥⎥⎦

T

·M · v =

⎡

⎢⎢⎢⎢⎣

3

2

5

4

⎤

⎥⎥⎥⎥⎦

T ⎡

⎢⎢⎢⎢⎣

5f1 + 7f2 + 9

5f3 + 7f4 + 3

5f5 + 7f6 + 2

5f7 + 7f8 + 4

⎤

⎥⎥⎥⎥⎦

= mi · v (3.20)

To complete the challenge, the client must compute the corresponding σi,j; this can be done bycomputing the σ values for each one of the basis vectors:

σ = M · v + r =

⎡

⎢⎢⎢⎢⎣

5f1 + 7f2 + 54

5f3 + 7f4 + 51

5f5 + 7f6 + 99

5f7 + 7f8 + 84

⎤

⎥⎥⎥⎥⎦

(3.21)

and then computing the dot product with the coefficient vector of node i, (3, 2, 5, 4) in the example.

Security for the construction follows from the framework definition, as long as the outputsfrom fk1 and fk2 are unpredictable. In the case of fk1 , we are generating random vectors tocompute a dot product; the probability of the function producing a particular value within thefield F is 1/|F|; this follows from the fact that in a field, if ab = ac then b = c, for a = 0.Then by multiplying a random value a, any outcome is equally possible, hence unpredictable fromthe attacker’s point of view. However, since a file can contain many zeroes, fk1 could becomepredictable; to prevent this, file pre-processing techniques such as: a file representation withoutzeroes (e.g. replacing zero by two consecutive appearances of a given value) or encrypting byadding a random number to each file element, can be used to guarantee this property (the encodingis reversed when the client needs to use the file). Regarding fk2 , we are adding a random numberwhich is guaranteed to be unpredictable from the attacker’s point of view; therefore, the functionfulfills the requirements to be considered within the framework presented in [49].

3.3. PROPOSAL 65

This construction can also be used by the new regenerated nodes, to check that the informationthey are receiving is not polluted. In this case the file owner, gives the secret vectors to the newnodes, such that they can verify the incoming information. For this particular case, the verifieris the node that is being regenerated and the provers are the nodes providing information for theverification. Now we will present some strategies to reduce the number of authenticators that thefile owner must compute.

3.3.3 Key Management Strategy

We previously described a system for one node and l vectors v(i). When the system consists ofn storage nodes, several key management strategies could be implemented. The first of theminvolves asigning l independent vectors v(i) to each storage node, the problem is that this strategyinvolves O(l · n) computation overhead as well as storage proportional to O(l) at each storagenode. A variant of this strategy would be to compute only l different PORs and query morethan one server simultaneously. However, one must not reuse the same vectors v(i) for tests notperformed simultaneously; otherwise a coalition of servers with the capacity to regenerate a givennode, could compute the result for this node in a descentralized way and store the intermediateresult µ′. Once µ′ is available, the POR test can be passed without having access to the file.

The previous remarks about reusing vectors, introduce us to one of the subtleties for keymanagement in PORs for encoded files. Since data stored is highly redundant thanks to the re-generating code used for encoding, any d ≥ k nodes can compute the right σ′ value, because anyd ≥ k has the capacity to reconstruct the entire file. For this reason, this kind of primitive mustnot be used in systems where storage nodes are likely to collude; otherwise, colluding nodes couldcreate the false impression of redundancy which is not desired from the file owner’s point of view.

Having this inherent constraint in the system, we can devise better key management schemes.The second option is to use a key assignment that gives us the guarantee that if the number ofnodes sharing their vectors in the network is less than c, there are at least d secrets unknown byall the nodes in the network. Such a construction can be made efficiently using a (c, d)-cover freefamily. Before defining it, we will introduce some sets: Let K be a set of cryptographic keys andB = (K1, . . . ,K|B|) be an ordered set of subsets ofK .

Definition 1. The system (K,B) is called a (c, d)-cover free family, if for any c distinct sets(K1, . . . ,Kc), any other set Γ ∈ B not in the collusion satisfies

∣∣∣∣∣∣

Γ \c⋃

j=1

Kj

∣∣∣∣∣∣

≥ d (3.22)

In practical terms, the previous definition means: it is possible to assign the authenticatorsto the nodes during regenerations (or initialization), in such a way that the collusion of up to c

compromised nodes, does not know at least d secrets from the total set of authenticators computedby the file owner. The effect of this, is that malicious nodes cannot send polluted packets to othernodes without being detected.

This kind of system can be constructed using polynomials, as shown in [80]. By using thiskey assignment, the authors can guarantee that their system can withstand up to c attackers having


at least d secrets unknown to them. This kind of key assignment was used in a protocol to preventpollution in Network Coding in [31]. The polynomial construction can be made proportional toO(l·√n) in both computation and space, which is more efficient than the previous two alternatives,

at the cost of small tolerance to colluders which is in the order of√n. It is worth noting that

variables c, d and n are not independent of each other, the bounds we provided were selected tomake the system perform well in terms of space and computation.

Even though the previous construction is significantly more efficient than the basic ones, it is ageneric construction which does not make any use of the linear properties of our system. For thisreason we introduce a simpler system which can tolerate up to k − 1 attackers, regardless of thenumber of users, at the cost of larger storage per node. This key assignment is meant to be usedamong nodes for pollution detection among them, but is insecure when used among storage nodesand the file owner.

The construction consists in generating all the secret vectors using a basis of column vectorsV = {v(1), . . . , v(k)}. Vectors assigned to nodes are of the form v ∈ Span(V ) and are producedas a linear combination of all the vectors in the basis. These vectors can be used to compute theproduct Mv where M is a matrix representation of the file with the right dimensions. Given acollusion of k − 1 or less attackers it is still difficult for them to produce forgeries. From the fileowner’s perspective (who has the entire basis), finding the right result can be achieved.

3.3.4 Protocol

The protocol starts, by modelling the original file as a matrix. The number of columns of thismatrix will determine α (storage at one node). The first part of processing involves applying thePORs from section 3.3.1 or 3.3.2. Then, the regenerating codes are applied over the values of asingle column, for all columns of the matrix and assigned to each storage node. The number ofrows of the matrix is the minimum number of storage nodes needed to reconstruct the file. Theprocess is described in Fig. 3.2 with a sample file of length 8 (in case the file is larger we caniterate the process).

Since vector σ containing the output of the POR is not coded for the unbounded-use POR, itis necessary for the owner of the file to have a way to know what vectors were used to create thedata stored by a node; using this vector, it is possible to reconstruct the right value for that givenvector given the linear properties of the function. If the vector is stored at the server, it must beauthenticated and encrypted, otherwise the system becomes insecure. For the bounded-use POR,vector σ must also be authenticated related to its position; to prevent attacks that involve swappingthe authenticators; in this case coding information is received from the server as well.

Finally, the modified pollution detection routine is computed and secret vectors are sent to thestorage servers.

To check if the coded files are available at the server, the file owner must run the verificationprotocol for the POR. This guarantees the servers are storing information according to the linearcombinations of the file without downloading the files. In addition, the file owner must check thefile can be reconstructed from the linear combinations advertised by the storage nodes. This isdone by simulating file reconstructions and node repairs for all possible failure scenarios, using

3.3. PROPOSAL 67

m1,1 m1,2 m1,3 m1,4m2,1 m2,2 m2,3 m2,4

σ1

σ2

c1,1 c1,2 c1,3 c1,4c2,1 c2,2 c2,3 c2,4c3,1 c3,2 c3,3 c3,4

σ′

1

σ′

2

σ′

3

File in ma-trix form

Apply regen-erating codecolumn-wise

Store row-wise.Node i storesci,1, ci,2, ci,3, ci,4

Apply modifiedPOR row-wise

Figure 3.2: Procedure to store a file of length 8 as 2 sectors of length 4 each; the information willbe stored at 3 storage nodes. The modified POR is applied to the rows of the matrix, while theregenerating code is applied over the columns. Given the properties of the modified POR, it ispossible for the owner of the file to run the protocol over coded versions of the file. The outputof the PORs must be unknown to the servers in the case of the unbounded-use one and availableto the owner during the verification protocol. For the bounded-use one, tags can be stored at theserver by adding an authentication function specifying to what test they belong.

the coefficients of the regenerating code. For instance, continuing with the example on Fig. 3.2,assume the regenerating code stored the following combinations

⎡

⎢⎣

c1,j

c2,j

c3,j

⎤

⎥⎦ =

⎡

⎢⎣

2 1

1 2

1 1

⎤

⎥⎦

[

m1,j

m2,j

]

. (3.23)

Then, what needs to be checked in that case, is that all combinations of k = 2 rows from the coef-ficient matrix, can reconstruct the entire file and regenerate a failed node. After this, regenerationcoefficients may differ from the original ones in functional regenerating codes, but for a correctsystem, this property still holds.

During regeneration of a failed node, cooperating nodes first send to the replacement node, thecoefficient vectors that they would get as a result of computing the regeneration routine. Next, thereplacement node must check the combination of received vectors in fact gives a good replacementas a result; this is done by checking the properties of the coefficient matrix. After this, cooperatingnodes send the actual payload, which is checked using the pollution detection routine. To finishthe process, the replacement node executes the regeneration routine.

3.3.5 Guaranteeing Decodability in Polynomial Time

Even though coefficients for the linear combinations could be selected randomly, it is better toguarantee the encoding matrix has a given form to check its properties efficiently. This is im-portant, because it allows any node in the network to check the system satisfies the k-node-reconstruction property, in time linear to the number of nodes rather than exponential.

Consider a network coding matrixM with n rows and k columns, each row ofM starts withk coefficients stating what combination of original vectors was used to create this row, followed


Algorithm 4 Verifying the k-node-reconstruction property in linear time.1: function VERIFY(M : The coded file matrix)2: for each row r ∈M do3: a← 1/r04: for each coefficient c ∈ r do5: if c = 1/a then6: return false7: end if8: a← a+ 19: end for10: end for11: return true12: end function

by the encoded payload. In our construction, checking that the k-node-reconstruction property issatisfied, is equivalent to checking any combination of k rows in M is linearly independent. Astraightforward argument shows, there are

(nk

)

possible combinations.To make the algorithm linear in the number of rows of the matrix, we need to be able to

check the matrix has this property efficiently without evaluating all the combinations. Fortunately,matrices such as Vandermonde or Cauchy have this property.

We will use a Hilbert matrix, which is a special case of a Cauchy matrix; its construction is asfollows:

Hij =1

i+ j − 1H =

⎡

⎢⎢⎢⎢⎢⎢⎣

1 1/2 1/3 1/41/2 1/3 1/4 1/51/3 1/4 1/5 1/61/4 1/5 1/6 1/71/5 1/6 1/7 1/8

⎤

⎥⎥⎥⎥⎥⎥⎦

(3.24)

Since we know this kind of matrix has the required linear independence property, the verifi-cation routine simply needs to find H−1

i1 which gives the row of the received vector within theoriginal Hilbert matrix; then, we verify the rest of the row matches the given row of the Hilbertmatrix. This must be done for the first k values of the row, which represent the coefficients usedto code the original information. Since we only need to check a coefficient element for each nodeonce, the total complexity of the procedure is O(k · µ), where µ denotes the cost of a singlemultiplication operation. Algorithm 4 illustrates how to do it on matrixM .

It is worth noting at this point that regeneration for our code is functional. For this reason,strategies must be used in order to be able to assign a particular linear combination during theregeneration stage to a server; otherwise the entire file must be transmitted to regenerate a singlenode. The most straightforward strategy to achieve this, would be to have a systematic copyavailable at a single server; this copy could be used to generate the desired linear combination. Ifthe systematic server fails, it could be reconstructed using the encoded servers.

A generalization of the previous approach comes from the Twin Code Framework [81]. Inthis framework, storage nodes are separated into two types. Nodes from one type store the output

3.3. PROPOSAL 69

Node A Node B Node C Node Dm1 m2 m3 m1 +m2 +m3

m4 m5 m6 m4 +m5 +m6

m7 m8 m9 m7 +m8 +m9

Type-0 nodes

Node E Node F Node G Node Hm1 m4 m7 m1 +m4 +m7

m2 m5 m8 m2 +m5 +m8

m3 m6 m9 m3 +m6 +m9

Type-1 nodes

Figure 3.3: In the twin-code construction from [81], a file is represented as a matrix M . Next,an erasure code is applied to the matrix to create nodes of type-0; to create nodes of type-1 weapply an erasure code toMT (T is the transpose operator). As a result, nodes of one type can beregenerated using the erasure code from the other type. Here node B of type-0 can be regeneratedby receiving any 3 elements from the second row of the matrix corresponding to nodes of type-1;which in this case are given by nodes E, G and H.

of the data in matrix form using a (n, k)-erasure code; for the second type, the data matrix istransposed and a second (n, k)-erasure code is applied. In other words, their construction embedsthe file in a matrix and applies an erasure code over the rows and another code over the columns.Nodes containing the output of the first code are called type-0 while the other ones are calledtype-1 nodes.

The result of this construction is that data stored by a given node can be reconstructed usingnodes from the other type. In Fig. 3.3, it is possible to see that the i-th column of the matrixformed by type-0 nodes, can be reconstructed using the i-th row of the matrix formed by type-1nodes; this property holds because of the erasure code. The converse is also true for columns ofthe type-1 matrix.

In the example of Fig. 3.3 a file M = (m1, . . . ,m9) is encoded according to the scheme, italso shows how Node B could be recovered using type-1 nodes. In practical terms what occurs isthat the second value stored by type-1 nodes, is a linear combination of the elements m2,m5,m8

of M ; since these are exactly the values stored by Node B, it is possible to recover the originalvalues by knowing the coefficients of the combinations stored by type-1 nodes. The combinationinformation can be obtained from the particular description of the code. For files longer than 9,the file is processed 9 values at a time.

The effect of the previous framework is that we can simulate exact repair for this code andkeep the assignment based on the Hilbert matrix. In case more nodes are needed, they can becreated transmitting the minimum amount of information possible. This property is very usefulto achieve load balancing using Regenerating Codes. By selecting nodes of one type as nodes tocreate new copies and nodes of the other to serve different clients accessing online content (e.g.multimedia), it is possible to balance the load among the nodes containing encoded pieces.

Now we will prove some properties about the approach to select the coefficients.

Theorem 2. The verification routine is asymptotically optimal in time for network-coding-basedregenerating codes.


m1+m2 m3+m4

2m1+m2 2m3+m4

m1+2m2 m3+2m4

m1 m2

m3 m4

Systematic

Systematic

Non-Systematic

Figure 3.4: Creating non-systematic nodes

m1+m2 m3+m4

2m1+m2 2m3+m4

m1+2m2 m3+2m4

m1+m2 m1+2m2

m1 m2

m3+m4 m3+2m4

m3 m4

User receives User decodes Non-Systematic Nodes

Figure 3.5: User reconstructing a file from non-systematic nodes

Proof. Non-Probabilistic algorithms wishing to verify the properties for the coefficients of a ma-trix must read the coefficients themselves; thus the complexity of this problem is lower boundedby O(k2). The complexity of the verification routine for each row is given by O(k · µ) where µis the cost of computing 1/a. Since the values for the matrix are known during creation time, theycan be stored and accessed during the verification routine in constant time. This procedure makesthe complexity of the routine O(k2), which cannot be improved given the lower bound.

Theorem 3. The construction guarantees the k-node-reconstruction property for the original in-formation, when at least k non-systematic nodes are available.

Proof. The coefficients associated to the linear combinations belonging to node i are the same asthose of a Hilbert Matrix. By definition, any κ · k of these rows are linearly independent and canbe used to decode the original data units.

We will now take a look at how can the system be used in multimedia streaming.

Using the Twin Framework for Multimedia Streaming

An interesting scenario for our construction is multimedia streaming. The motivation for thisscenario comes from several reasons: on one side, multimedia information is large, making it hardto store entirely in a single machine; this makes it a good candidate for erasure coding as well.On the other hand, users do not need to download at the maximum speed possible, any transferrate above the bitrate of the content is enough for a good user experience. This makes it a goodcandidate to share the load at different servers simultaneously, in order to ease how a contentcatalogue is distributed across different servers.

3.3. PROPOSAL 71

We show an architecture based on the twin framework in Fig. 3.4 and Fig. 3.5. The idea is tohave nodes of two types, the first type called systematic contain contiguous sections of a file andtheir objective is to create new non-systematic nodes. Non-systematic nodes have as a missionserve the requests of the clients. The process of creating non-systematic nodes is illustrated inFig. 3.4, where each systematic node creates linear combinations of its stored pieces. Fig. 3.5shows the procedure of a client that wishes to access the content by downloading pieces fromseveral nodes. The code used in the Figure is not using our proposed assignment based on aHilbert matrix, but it illustrates the relevant parts of the architecture. From a client’s perspective,decoding can be achieved as soon as encoded pieces from enough nodes have been received, thisis done for small sections of the file to allow playback before the file has finished downloading.

3.3.6 An Application for Synchronizing Multimedia Collections

Cloud music services offer an interesting scenario where PORs can be used. Several online musicservices such as Google Play’s “scan and match” [82], offer their users the possibility of storingtheir music files in their servers. Users can then access the files from any device that can connectto the service. Taking advantage of the fact that some of the songs are already in the catalog ofthe music service, files that are deemed equal according to a matching process are added to theusers account automatically; only the files that cannot be matched are sent through the network.This reduces the network load on the system. It is important to mention that music services onlywant to bypass transmission when the user actually has a song, otherwise they would be receivinga catalog song for free, which translates into loss of revenue. This also provides motivation for anattacker to bypass the verification system to obtain songs without actually having them in the firstplace.

From a technical perspective, the process starts by creating a representation of the music file.Ideally, this representation should be an acoustic fingerprint such as the ones analyzed in [83],instead of a simple binary comparison of the two files. In this way, two files that represent thesame song but are in a different format, can be deemed equal. The second step involves comparingeach fingerprint independently against the music service catalog to find a match. Even though thefingerprint is smaller than the original file (see [83] Section 5.2 for a comparison), for larger musiclibraries in the order of thousands of songs, the overhead could still be far from optimal. From thesystems evaluated in [83], the one with lowest overhead requires 1467.5 bytes in average to createa fingerprint for 10 seconds of a song. Extrapolating this value to a 4-minute song, gives 34.39KBas a result.

Since each song is compared independently, the network overhead of this approach would begiven by the size of the audio fingerprint of each song, times the number of songs in the userlibrary. It is possible to perform this process regardless of the size of the audio fingerprint for eachsong, as long as we know the name of the song we want to compare. This assumption appliesto users, who might have created backup copies of their CDs or users who have downloaded thesongs from services with compatible names.

The idea of the protocol to reduce the fingerprint overhead, is to divide a song’s fingerprint indifferent subvectors and ask the client to provide the result of the bounded-use POR, for a randomlinear combination chosen by the music catalog. Since all the bytes of the fingerprint are involved


in the computation, the user can only pass the test if it knows the exact fingerprint of a song. Theprocess is as follows: let mu be the fingerprint of the song in possession of the user and mc bethe fingerprint of a song with the same name in the music catalog. The user divides mu in k non-overlapping vectors mu,i of size |mu|/k. Then, the music catalog sends a seed s to the user, togenerate k random values α1, . . . ,αk. The user now must prove that it has the value

k∑

i=1

mu,i · αi (3.25)

which can be done using the POR from Section 3.3.2. In this scenario the user is the one per-forming as prover and the music service as verifier. One advantage of using our bounded-usehomomorphic POR is that several songs can be combined into a single execution of the protocoldue to its linear properties, yielding a much more efficient solution when the names of the songsto be compared are known. However, if some songs differ, a search needs to be implemented toisolate the matching songs. The reason for using the bounded-use POR is that its overhead is lessthan the unbounded-use POR and our goal is to minimize the amount of transmitted information.In addition, in this scenario both the prover and verifier have access to the same file, which makesit possible to create an unlimited number of new challenges for the bounded-use POR.

Since we are aggregating several fingerprints into a single one, it might be the case that someof the fingerprints differ from the ones in the online catalogue. For this reason, the entire aggregatefingerprint needs to be rejected, and a new aggregate fingerprint must be submitted. For instance:assume the user library has s = 1000 songs and denote by f1, . . . , f1000 their fingerprints; now,assume that w = 10 of them do not match the ones in the catalogue. The problem then is to findhow to design tests to find the s − w = 990 songs in the database by using aggregations of theindividual fingerprints. This problem is the study area of a branch of combinatorics called “GroupTesting”, where a number of defective elements must be found from a set, while minimizing thenumber of attempts; it was born as a way to design combinatorial experiments for biologicaltesting [41].

In particular for this scenario, a branch known as “Competitive Group Testing” which consid-ers the results of previous tests and does not have any assumption about the number of defectiveelements, can be used. In our scenario, a defective element is a fingerprint the user has, that cannotbe matched to a fingerprint in the online catalog; hence, we have w = 10 defective elements. Con-sider our sample scenario with s = 1000 and w = 10, the naive approach would simply request allthe fingerprints independently for a total of s = 1000 tests. Now, consider the competitive grouptesting scheme presented in [84], which given s and w, has a worst case number of attempts lessthan or equal to:

1.65w(logs

w+ 1.031) + 6 (3.26)

This worst case operates regardless of where the w defective elements are. For our scenario withs = 1000 and w = 10, the worst case is 126.63 attempts before all the unknown songs arefound. Which is a reduction of almost 83% compared to the naive approach. When the number ofdefective elements increases, the number of attempts needed to find them also increases. For the

3.3. PROPOSAL 73

case of s = 1000, the construction from [84] can find up to 168 defective elements, with less than1000 tests; for more defective elements, the approach is less efficient than the naive approach. Fora summary of algorithms in this area, we refer the reader to Chapter 4 of [42].

The idea for aggregate tests could be used to further improve the efficiency of cloud musicproviders offering to store user’s music libraries. However, the success of an actual implemen-tation depends on being able to minimize the number of aggregate unknown values for a singletest. Regarding security, the protocol as it is does not prove that the user has a song, but actuallythat it has access to the fingerprint of a song, for this reason a secure implementation should alsoconsider adding some randomness to the fingerprint generation. We do not treat the creation ofsuch secure fingerprints in this work.

Another possibility for minimizing bandwidth usage, could be to simply transmit parts of thefingerprint. Despite its attractiveness, this approach suffers from one problem related to differentversions of the same song. In particular, users of some of these services, have found that theirversions of the songs with explicit lyrics, were replaced by the cloud provider, with the censoredversions of the same songs [85]. Since the censored version is an almost identical copy, such anapproach should consider this.

Another alternative could be to compute a cryptographic hash of a fingerprint, the problemwith this approach is that efficient hash functions do not have any homomorphic properties likeour constructions. For this reason, using a group testing approach would imply recomputing thefunctions many times. To clarify this point, consider the following simple group testing approachbased on bisections:

1. Let f1, . . . , f1000 be the fingerprints of the files to be tested.

2. Create one aggregate fingerprint using (f1, . . . , f1000) and add it to a queue.

3. Get the first aggregate fingerprint from the queue. If it can be matched, stop there andadd all the files to the user music library, without sending them through the network. Ifthe aggregate fingerprint does not match, then compute two separate fingerprints by usinghalf of the original fingerprints of the current one. For instance, if the current aggregatefingerprint was computed using (f1, . . . , f1000), then compute an aggregate fingerprint using(f1, . . . , f500) and another one using (f501, . . . , f1000), add both aggregate fingerprints tothe queue.

4. If the queue is empty stop; otherwise go to to step 3.

Instantiating this algorithm with our construction which has homomorphic properties is very con-venient. We simply need to compute an array M where position j is Mj =

∑ji=1 H(fi) where

H is the result of applying our proof to a fingerprint, also recall that the output of our construc-tion is an element from a finite field. If the group testing algorithm requires the output of theaggregate fingerprint for the original fingerprints (fstart, . . . , ffinish), the output can be computed asMfinish−Mstart+H(fstart). A function without this property would have to compute and store par-tial results; then, it would have to traverse the data store to reduce the number of operations. Thisis the case for cryptographic functions such as SHA, for which there is no easy way to subtract thecontribution of a given fingerprint in the aggregate fingerprint.


Another advantage comes from the actual number of aggregate fingerprints that need to besent through the network. If the aggreggate fingerprints for (f1, . . . , f1000) and (f1, . . . , f500) areknown, then the aggregate fingerprint for (f501, . . . , f1000) can be computed by a simple subtrac-tion. On the other hand a non-homomorphic function such as SHA, must still send the 3 aggregatefingerprints explicitely. Fig. 3.7, shows the savings achieved by this idea.

In the next section we will show that this proposal is asymptotically better than existing genericgroup testing schemes, to find defective blocks.

3.3.7 Improving the Bisection Method

Our proposal improves on the bisection method from Algorithm 3, our improved construction ispresented in Algorithm 5. The improvement comes from two changes:

1. We keep the result of the previous test fp as a parameter given to the function.

2. We use the homomorphic property of the PDP in line 10.

Given that in a homomorphic PDP the output for the first half of the elements f1 plus the outputfor the second half of the elements f2 equals the result of applying the PDP to the whole set fp, wenever need to call the ApplyTest routine to compute f2. This implies that regardless of the patternof defective elements, we can compute any combination of 2 defective elements with at most twotransmission from the prover, as shown in Fig. 3.6.

Algorithm 5 BisectHTRequire: elems: set of elements with at least one defective, fp: result of applying the PDP to

the parent nodeEnsure: D: Set of defective elements1: D ← ∅

2: if elems.length() = 1 then3: D ← D ∪ elems return D4: end if5: f1 ← ApplyTest(elems.first half())6: if verify(f1) is TRUE then7: D ← Bisect(elems.second half()) ◃ The defective element is in the second half8: else9: D ← D ∪ Bisect(elems.first half())10: f2 ← fp ⊖ f111: if verify(f2) is FALSE then12: D ← D∪ Bisect(elems.second half())13: end if14: end if15: return D

Fig. 3.6, presents the four possible outcomes for tests with two elements and how manytransmissions are needed for the Bisection and BisectionHT routines. For the case where the rootof the tree is clean, it automatically means that both leaves are clean, therefore no further tests needto be transmitted for both schemes; this gives a total of 1 transmitted token per scheme. When theleft child is clean but the right one is defective, both algorithms transmit two tokens. However, for

3.3. PROPOSAL 75

Number of transmissions for (Bisection, BisectionHT)

0

0 0

(1,1)

1

0 1

(2,2)

1

1 0

(3,2)

1

1 1

(3,2)

Figure 3.6: Number of tests for different configurations of two elements. The leaves of each treerepresent two possible elements 0 for clean, 1 for defective, the root node represents the result ofperforming a test with the two leaves as members of the pool. Each pair (a, b) under each treedenotes the number of results that need to be transmitted by the prover during the verificationroutine to find all the defective elements for the Bisection (a) and BisectionHT (b) routines.

the other two cases, the Bisection routine cannot distinguish them without transmitting the resultfor all tests, this is where difference in performance relies between the two routines.

In order to use the BisectionHT routine, the client performs an original invocation to the PDPasking for the aggregate result of c blocks, let fp be the name of this result. If the prover doesnot pass the test, then the BisectionHT function is invoked. If partial results are also kept at theserver, then computation of the tests during the search stage do not involve any disk access fromthe prover. From the verifier side, only the first half of each defective test must be submitted, thesecond half can be computed using the homomorphic property of the PDP scheme. The effectthis has in the overall transmission of the results from each individual test is summarized in thefollowing theorem.

Theorem 4. Let f be a function that produces a homomorphic token for a PDP verification pro-tocol, and f1, . . . , fs be the result of applying f to s different blocks where s = 2n, and at most0 ≤ d ≤ s of them are defective. Then, it is possible to apply Algorithm 5 on f to find the d

defective blocks, transmitting at most s linear homomorphic tokens.

Proof. The bisection method can be seen as a full binary tree with log2 (s) levels, where the rootis at level 0 and the leaves are at level log2 (s) − 1; the leaves are the result of applying f toeach of the s files. For each internal node P , its left child 1 represents the aggregate result ofapplying the homomorphic function to the first half of the files, and the right child 2 represents theresult of applying the homomorphic function to the second half of them. If a node representing anaggregate PDP token matches, then all the leaves reachable from the current node are correct withoverwelming probability due to the properties of f . Since more invocations are equivalent to keeptraversing the tree, then the worst case scenario of the algorithm is when all the leaves have thewrong value.

Given the homomorphic property of f , we have the following relationship among the nodes inthe binary tree

fP = f1 ⊕ f2 (3.27)

therefore, once the server has received fP , it only needs to receive f1, the value for f2 = fP ⊖ f1.Where ⊕ and ⊖ represnt addition and subtraction, respectively.


Table 3.1: Times for the pollution detection routine

Size (MB) FieldGF (28) GF (232) GF (2147483659)

3.11 191 ms 157 ms 154 ms10.49 654 ms 538 ms 481 ms46.29 2364 ms 1980 ms 1779 ms105.97 7274 ms 6207 ms 6027 ms283.76 17655 ms 14786 ms 14381 ms

We will proceed by induction on the number of leaves, given an L(0) = 20 = 1 elementfull binary tree, we simply sent its value to the server, making T (0) = 1 transmission for onenode. Now, assume that for a tree with L(n) = 2n leaves, T (n) = 2n tokens are sent to theserver. Because of the homomorphic property, one token is transmitted to the server for each ofthe L(n) = 2n parents of the L(n+1) = 2n+1 leaves in the new tree. This makes the total numberof transmissions for the new tree

T (n+ 1) =T (n) + T (n)

=2n + 2n = 2n+1 = L(n+ 1). (3.28)

It can be seen that the number of transmissions is not greater than that of leaves which correspondsto the number of the homomorphic tokens sent through the network in the worst case.

It is important to note, that computing the homomorphic step, even though more efficient thanrecomputing the entire PDP for that subset, still requires some computation from the verifier.


The simulation environment for the test was an Intel Core 2 Quad Q6700 running at 2.67 GHz,with 8 MB in cache and a bus of 1066 MHz; the development environment was Java JDK 1.7

release 2.Tests involving the pollution detection routine were performed by reading the entire file in

memory before processing; this was done to minimize I/O factors from the simulation. Simulationwas performed using the SHA1PRNG random generator from the JDK. Regarding the finite fields,a custom implementation for 3 different finite field sizes 28, 232 and 2147483659; the implemen-tation for GF (232) was done by concatenating the implementation from GF (28), this allows toperform multiplications faster because it was precomputed for GF (28). Results for the simulationare shown on Table 3.1.

As it can be seen on the simulation, using this construction for large files is practical; even forour implementation that does not use any kind of hardware optimization. Other schemes in theliterature, propose the use of functions that guarantee public verifiability such as [77]. However,to achieve public verifiability, these systems perform many modular exponentiations, which areconsiderably slower than dot products as in our case.

To avoid computing the function many times, we experimented with a key assignment basedon a (c, d)-cover free family based on a polynomial from [80]. In practical terms it makes the


Table 3.2: Times for computing unbounded-POR tags for different block sizes

Size (MB) Block Size1 16 100


number of secret vectors s depend on the number of compromised servers, rather than just theserver themselves. Hence, a construction with a total of 32 = 9 vectors, could be used in a storagenetwork containing 34 = 81 servers, by assigning 3 secret vectors to each server, such that thecollusion of any 2 nodes does not suffice to know all the vectors of an honest node. Using ourproposed key assignment where vectors are combined has the same execution times, because 9

independent vectors are used; the benefits are that it works for unlimited nodes and tolerates up to8 compromised nodes.

To follow the description of the unbounded-POR implementation, we selected the prime fieldGF (2147483659). Execution times to generate the tags are summarized on Table 3.2. In ourimplementation calling the PRF was a very slow step; this explains why increasing the block sizeimproves performance. Arithmetic was performed using 64-bit longs.

It is worth noting that tags must be transmitted during the verification process; the overheadinduced by these tags is linear in the amount of tags, which for the s = 100 case, corresponds to1 % of the data stored by a node. Execution times for the verification procedure are in the sameorder of magnitude as tag creation times. The same overhead is needed for each of the secretsvector used for pollution detection.

To put the previous results in perspective, a system using the unbounded-POR with the vectorcombination key assignment, could be created with unlimited storage servers, each one having1 secret vector that could withstand an attack of at most 8 corrupt byzantine servers during theregeneration stage. The owner would still be able to detect the attack given the POR propertiesand would have to ask manually to regenerate nodes. The total transmission overhead induced bythe proposal using sectors of size 100, would be around 4% for the tags; this is caused by the factthat it is imperative to store the counter and the authentication information which are needed forall σ values and not just the output of the function. In addition, the secret vector would involvestoring an additional 1% of the file at each storage node.

We also simulated the bounded-use-POR; results are summarized on Table 3.3. Executiontimes for this test are similar to those of the efficient pollution detection routine, since they areboth based on applying dot products. For this test scenario, we assumed 5 nodes were needed toreconstruct the file, the effect this has on the execution times is that it reduces invocations to thePRF by 1/5.

In terms of overhead for this particular test, at the client it consists of 25 additional coefficientsto represent the linear combinations stored by each node; two keys k1, k2 of 128 bits, a hash ofthe original file 160 bits and 5 field elements. Overhead at servers consists of 5 σ elements andauthentication information for each one of them. The previous data applies for all the file sizes


Table 3.3: Times in ms for computing bounded-POR tags for different field implementations

Size (MB) FieldGF (256) GF (232) GF (2147483659)


Table 3.4: Qualitative comparison of the proposed PORs

Unbounded-use Bounded-useOperations over sectors entire fileNumber of challenges unbounded boundedTag gen. time small largeTag storage overhead large small (in practice)Challenge comp. time small largeChallenge overhead large small

of Table 3.3 since it does not depend on file size. We can see that the overhead is constant forthis scenario, compared to the approximately 4% overhead from the unbounded-use POR usingsectors of size 100. Table 3.4 has a condensed comparison about the characteristics of both PORconstructions.

The final simulation we performed was meant to test the performance of our constructionfor the application related to the synchronization of music libraries. On the client side, for asingle test our proposal was in average 8.3 times slower to compute on our test machine, thanthe default implementation for the SHA-1 function available at the Java JDK. Even though ourproposal is slower to compute for the client, the number of aggregate fingerprints that must be sentthrough the network is less than that of traditional approaches not using a homomorphic function,as shown in Fig. 3.7. It is important to note that given the linearity of our function, we need onlyperform the computation of the protocol once; for each subsequent bisection we simply performthe right addition using the linearity property. In contrast, an algorithm based on SHA where mustrecompute the hashes at each bisection step.

Fig. 3.7. compares three schemes: the first is the naive scheme, which consists of testing all thefingerprints independently; for this reason the number of transmitted fingerprints does not changewith the number of unmatched songs. The other two algorithms perform group testing based onbisection as explained in the previous section. The difference between the proposal and the SHAbased algorithm is that for the proposal, whenever there was an aggregate fingerprint that couldnot be matched, only one of the smaller aggregate fingerprints was sent through the network. Thevalues that were plotted correspond to the maximum number of transmitted aggregate fingerprintsfor a particular number of defective elements. Each scenario for a given number of defectiveelements was simulated 100000 times for a user multimedia library consisting of 1000 files.

In terms of bandwidth savings in our simulation, our proposal was able to find all the un-matched songs w by sending less than 1000 aggregate queries, for all the possible values of w,confirming the results of the theorem. In contrast, the bisection method using the SHA-1 hash

3.5. CONCLUSIONS 79

0

200

400

600

800

1000

1200

1400

1600

1800

2000

0 100 200 300 400 500 600 700 800 900 1000T

ran

smit

ted

ag

gre

gat

e fi

ng

erp

rin

tsUnmatched songs

Transmission of Aggregate Fingerprints

SHA-MaxProp-Max

Naive-Max

Figure 3.7: Simulation depicting, the maximum number of aggregate fingerprints needed to findall the unmatched songs in a given collection. Simulations for each number of unmatched songs,were performed 100000 times. The size of the collection in the simulation was set to 1000 songs.

function already had worse performance than the naive approach, when w = 150. For the rangeof unmatched songs where applying the bisection method was better than the naive approach, ourconstruction was able to find the unmatched song, sending about 50% less aggregate fingerprintsthan the SHA-1 variant.

3.5 Conclusions

We presented in this chapter, a practical scheme that stores a file at a set of servers using a Regen-erating Code. The system is able to guarantee a particular node is storing a linear combination ofparticular data units, using the proposed key management schemes for pollution detection and thetwo proposed POR constructions. In addition, it is also possible to verify in polynomial time thatthe encoded pieces can reconstruct the original file.

In terms of the presented bounded-use vs unbounded-use PORs, the bounded-use constructionhas better overhead than the unbounded-use one, it also incurs in less overhead per challenge.However, in terms of computation, it has a much larger storage overhead when information needsto be queried frequently for a long time. Simulations showed the system is computationally practi-cal and has reasonable overhead, which can be adjusted to conceal the requirement among security,frequency of PORs and amount of storage. As an additional application of our scheme, we pre-sented a new application for PORs to synchronize libraries among users and cloud music services.We show how our proposal can improve the performance of a group testing method based on bisec-tion, which translates in bandwidth savings for users as well as servers for the proposed scenario,regardless of the number of elements that are not matched.

Chapter 4

Diversity and Decodability Attacks

81

82 CHAPTER 4. DIVERSITY AND DECODABILITY ATTACKS

4.1 Introduction

This chapter will discuss our contributions towards the problem of Diversity Attacks in transmis-sion and storage networks, as well as Decodability Attacks which occur in Immediately DecodableNetwork Coding (IDNC). It will provide our constructions to address the challenges presented inSection 1.3.3.

Diversity Attacks were proposed by Popa et al. in [7]. Their goal is to reduce throughput bynot coding from all the source packets received by the attacker. This is a very interesting attack,because it does not rely on the integrity of a single packet, as it is the case with pollution attacks,but also on factors related to the topology of the network. The solution proposed by Popa et al.is very elegant, it proposes adding the relevant parts of the network coding protocol to the packetcreated by the intermediate coder node. Then by using a special type of digital signature withlinear properties, each verifier can recreate the computation and to relate the contribution of eachparent node to the encoded packet. The pattern for this solution is common in verifiable computingwhere a computation is outsourced to a third party while maintaining verifiable results. For thisparticular scenario, there is a need to prove what inputs were used to create the packet usingnetwork coding. The drawback of their construction is that they explicitly authenticate all theinputs to the coder node, and this information is transmitted to the verifiers (children). This createsa high transmission and computation overhead in the form of the authentication information.

A different scenario for Diversity Attacks arises in storage networks. Some storage networksuse the idea of allowing intermediate storage nodes to apply transformations to their data to re-generate nodes that have failed. This strategy has been used to create a type of codes known asRegenerating Codes [9]. Their goal is to reduce the amount of bandwidth needed by erasure codesto repair a node that failed. Similar to the transmission scenario, some storage nodes which canbelong to different storage services (e.g. companies), might try to provide corrupt informationto a new node in the network. The motivation behind such attack would be to make a particularserver to seem more reliable than others. This in turn could attract more customers and revenueto the company. To the best of our knowledge we are the first to address this problem for storagenetworks, without the file owner needing to see the contents of the file.

Decodability Attacks proposed by Corena et al. [8] have some similarities to Diversity Attacks.This attack occurs in IDNC, where nodes cannot perform buffering to decode packets later. Thegoal of the attacker then, is to select packets that can be decoded by the least amount of nodes ina network, decreasing the overall throughput. To prevent such attacks, we need to know that thenode is actually taking a reasonable decision based on the packets that it has received. Making thisattack an instance of verifying inputs and outputs in Network Coding, similar to Diversity Attacks.We call this kine of security problems depending Verifiable Network Coding.

Despite their similarities, Diversity Attacks and Decodability Attacks are not the same attack.As illustrated in Fig. 1.10, including information from more nodes into a packet, does not neces-sarily help throughput in IDNC, in fact it can actually make it worse.

The contributions of our work related to these topics are the following:

1. Present the first scheme to prevent diversity attacks that can adapt to dynamic topologiesand has the lowest transmission overhead for non-randomized detection systems.

4.2. PROBLEM AND NOTATION 83

2. Transmission improvements to Popa et al.’s Payload-Independent Protocol (PIP) [7] usingaggregate digital signatures.

3. Present the first mechanism to defend against our proposed problem of decodability attacksin IDNC.

4. Present a construction to prevent diversity attack in storage networks using RegeneratingCodes.

4.2 Problem and Notation

“Diversity Attacks” were defined by Popa et al. in [7], we will make use of a notation similar totheirs. A source S transmits packets to a set of sinks D; each packet has the form (M,C) whereM denotes the payload and C denotes the appended coded vectors used for decoding. Both M

and C are vectors of elements in a finite field of size q. As the packet travels the network, we willconsider node N, who performs random linear network coding over a set of packets transmitted bya group of nodes PN (N’s parents); the output of N will be transmitted to a group of nodes CN (N’schildren). To perform random linear network coding over a set of packets EPj = ⟨Mj , Cj⟩, nodeN outputs:

EPN =

⎛

⎝

∑

j

αj ·Mj,∑

j

αj · Cj

⎞

⎠ (4.1)

where each of the αj is a randomly selected coefficient from the same field as the packet andcoding vectors.

The goal is to give nodes in CN a protocol that allows them to check N is coding using aparticular subset of PN. We assume nodes can overhear each other’s packets, which is usually thecase for network coding broadcast transmission protocols, we also assume that packet may be lostduring transmission. Also, nodes have a way to authenticate messages among them, and in casemisbehavior is detected, N can be excluded from the network or punished somehow in a protocoldependent fashion (e.g. nodes in CN do not forward N’s packets anymore). All these assumptionsare reasonable and have been used in the literature before: overhearing has been employed in [86]to detect misbehavior; and an efficient broadcast authentication protocols can be found in [62].

4.3 Cryptographic Preliminaries

4.3.1 Digital Signatures

Digital signatures in their usual context, allow any node in possession of a public key Pk, checkthat a node in possession of the corresponding private key Sk, generated a message m; however,the knowledge of Pk does not allow nodes to produce valid digital signatures; sample construc-tions include RSA [61] and DSA [87]. In a (t, l)-threshold signature scheme [88], the privatekey Sk is divided in l shares, such that the collusion of any t of produces a valid signature. This


Table 4.1: Notation summary

Symbol DescriptionN Node performing codingPN Parents of node NEPj Packet sent by node jσS(EPj) Pollution signature of EPjνS(EPj) Verif. pollution for EPjχ Combination for pollution signatureσPj(EPj) Authentication for EPj by PjνPj(EPj) Verif. of authentication by Pj for EPjσ′Pj Approval signature by Pj

ν ′Pj Verif. of approval by Pjχσ′ Combination time for approval signatureh Preimage resistant hash functionq Size of the finite fieldαj ,βj random coefficients in Fq

τ Signed token containing (αi,σS(EPi),N)!(t) Any given value at a time t

signature can be checked by those holding the public key Pk, after the individual outputs of thefunction have been computed.

The following construction called Mediated RSA [89], is a (t, t)-threshold signature; wepresent the construction for t = 2: compute n = pq, where p, q are prime numbers of a suit-able size; then, compute φ(n) = (p − 1)(q − 1). Select e such that gcd(e,φ(n)) = 1 and d suchthat ed ≡ 1 mod φ(n); (e, n) is the public key. Now we randomly select two numbers d1, d2such that d = d1 + d2 mod φ(n), in selecting these numbers one must be careful not to revealnumbers that could be found by attacks on RSA (see [90] for a survey) ; give (d1, n) to signer oneas its private share and (d2, n) to signer two. When a message 0 < m < nwants to be signed, eachsigner computes σi = mdi mod n; these signatures are combined to produce the final signatureby computing

σ1 · σ2 ≡ md1 ·md2 ≡ mod n

m(d1+d2 mod φ(n)) ≡ md mod n (4.2)

the result is the same as in traditional RSA. Verification can be done by checking that m =

((σ1·σ2)e mod n). One important property about this scheme, is we can have as many groups of tindependent signer groups, without compromising its security. Therefore, if the number of signersis large, we can group them in groups of t nodes, such that unless t nodes from a group cooperate,they cannot produce valid signatures; however, if t nodes belonging to a same group collude, theycan find the private key. As a final comment on threshold signatures, there are construction whereparameters k and l can be selected at will such as [88]. In practice, m is not signed directly, but afunction of it. However, the previous operations operate exactly on the output of that function.

Some digital signature algorithms support batch verification. This means that it is possible toverify several signatures from the same signer on different messages simultaneously. This type ofverification should guarantee that all the messages were signed. Several versions of this technique


have been developed for RSA, including: Batch RSA [91] and Condensed RSA [92]. In Batch RSA,a verifier who receives signatures σ1, . . . ,σl on messages m1, . . . ,ml computes the following:

σ =l∏

i=1

σi mod n

m =l∏

i=1

mi mod n. (4.3)

The signature is correct if σe = m. Note that a single exponentiation needs to be performed forthe l signatures. In Condensed RSA, the verifier does not receive the individual signatures, onlytheir product σ.

An extension to the batch-verification scenario is possible, creating what is known in the lit-erature as Aggregate Signatures. In this type of signatures, l different signers sign l differentmessages. From a security perspective, an aggregate signature should convince the verifier thatthe i-th signer signed the i-th message. An instance of this kind of signature is the following con-struction presented in [93]. Let G,GT be cyclic groups with the same number of elements q, andcomputing discrete logarithms is hard. Let e : G×G→ GT be a bilinear map. A bilinear map isa function such that for u, v ∈ G, a, b ∈ Z

e(ua, vb) = e(u, v)ab. (4.4)

In addition to the previous, we require e to be efficiently computable and non-degenerate: e(u, u) =1 for u = 0. The instantiation for G is usually an elliptic curve and for GT is a finite field. Therelevant part of this selection, is that given u, ua, ub ∈ G, it is difficult to find uc for c = ab

mod q. This is known as the Computational Diffie-Hellman Problem (CDH) [94]. However,given u, ua, ub, uc ∈ G, the bilinear map allows us to verify whether a ≡ bc mod q by com-puting e(ga, gb) = e(g, gc). This problem is known as the Decisional-Diffie-Hellman Problem(DDH) [95]. Groups with this property where CDH is difficult but DDH is easy are known as gapDiffie-Hellman.

The parameters of the aggregate signature algorithm consist of the groups G,GT , the map e,a generator g of G, as well as a hash function H which maps inputs to elements of G. These pa-rameters are shared for all the signers. To generate a private key, each user selects a secret numberx ∈ Zq, the public key is computed as v ← gx ∈ G. For signing a messagem, the signer computesσ ← H(v,m)x ∈ G. A signature is deemed as valid by a verifier if e(σ, g) = e(H(v,m), v). Toaggregate several signatures σ1, . . . ,σl on messages m1, . . . ,ml from several signers, the aggre-gator computes σ =

∏li=1 σi. Note that the order in which the signatures are aggregated is not

important. The aggregate signature σ is valid if e(σ, g) =∏l

i=1 e(H(v,mi)).

Another kind of signature that will be needed for our purposes is a “Pollution Signature”; thiskind of signatures are meant to prevent packets not sent by the source to pollute the network, whichin the context of network coding translates to authenticate the linear subspace V spanned by thebasis vectors v1, . . . ,vn. In [30] Boneh et al. define linear homomorphic signatures for networkcoding as: given a set of signatures σ1, . . . ,σn for each of the basis vectors to be sent and a set of


weights (β1, . . . ,βn) ∈ Fq; then the following is a valid signature message pair:

(

σ =∑

i∈m

βi · σi,m =l∑

i∈m

βi · vi

)

(4.5)

A forgery is declared for this kind of signature, if a signature is created for a previously unseenvector space. An example of this kind of construction can be found in [30].

4.3.2 Aggregate Message Authentication Codes (MACs)

MACs are attractive for network protocols because they can be created using a fast Pseudo RandomFunction (e.g. HMAC) here authenticate means any of the parties in possession of the secret couldhave created it, hence they do not provide non-repudiation. Despite this drawback, MACs are veryattractive for network protocols since they are based on faster primitives, given that they can becreated from a fast Pseudo Random Function (e.g. HMAC).

One important property about MACs [96], is that vMACs F (ki,m), computed using differentkeys ki, on the same messagem; can be securely aggregated into a single MAC by computing:

F v =t⊕

i=1

F (ki,m) (4.6)

This property allows the construction of aggregate MACs and (t, l)−threshold MACs [97].When a verifier who knows all the keys ki, wishes to verify that a particular set of v nodes

participated in the creation of aggregate MAC F v, it simply computes a MAC F v∗ itself using thecorresponding keys. Whenever F v∗ = F v then it outputs 1, or 0 otherwise.

The reason for using aggregate MACs instead of threshold MACs is that, given the lack ofalgebraic structure, it is necessary to recompute all MACs at the verifier. By giving each signer aunique secret key shared with the verifier, any subset of signers can produce an aggregate signature.As a result, signers only need to compute one MAC, while the verifier minimizes its load bycomputing one MAC per signer.

4.4 Existing Work on Diversity

We will now present the existing work for the diversity problem and Linear PDPs.

4.4.1 Payload-Independent Protocol

The authors of [7] propose an elegant payload-independent protocol (PIP). This protocol uses theaggregate pollution signature σS(EPN) for the current packet, in conjunction with authenticatedtoken

τ = ∆,σPi(∆), (4.7)

where ∆ = (αi, Ci,σS(EPi),N). This token specifies the packet was sent to N, the coefficientαi that should be used over the set of coefficients Ci of packet EPi and the pollution signature

4.4. EXISTINGWORK ON DIVERSITY 87

Algorithm 6 Payload independent verification function at Cj1: for each parent Pi of N do2: check (∆i,σPi(∆i)) is present and valid.3: check αi = 0.4: check EPN is not polluted by verifying σS(EPN).5: end for6: check whether coefficients in EPN =

∑

(αi · Ci)7: check whether σS(EPN) =

∑

(αi · σS(EPi)).

σS(EPi) sent by each parent. Then, N creates a packet with the following format:

[EPN ||σS(EPN)||τ1, . . . , τ|PN|]

which consists of the coded packet, a pollution signature for the coded packet, and the coefficientsand pollution signatures of the packets from the parents of N, signed by each parent. Note that thepayload is not included, just the authentication signatures, pollution signatures and coefficients.

When a packet is received by any child Cj, Algorithm 6 is applied. The idea is that using thehomomorphic property from the pollution signature, Cj checks the resulting packet could haveonly be created using the packets from PN. The reason for security is that, given the securityproperties of a pollution signature; if node N is able to find a signature that matches, it must haveused the given packets.

To further reduce the overhead, they provide a probabilistic construction with constant over-head, which they call log-PIP. Their construction uses a Merkle tree to commit to all the τ valuesused. Then, parts of the tree are transmitted to allow nodes to check a particular node payload wasincluded.

4.4.2 PDPs with Linear Properties

The idea for this construction is to compute a set of values at the client in advance over the originalfile, then upload the file to a server while keeping the resulting values locally or protecting themat the server using encryption. This particular proof must be able to be compatible with the linearoperations performed by the encoding process. In other words: given a particular linear combina-tion of a set of vectors, the proof must be able to predict the correct result from the original file. Amore thorough explanation was given in Section 3.3.

We briefly recall the main idea about the bounded-use linear PDP described in Section 3.3.2for a single test.

1. Model the file to be stored as the rows of an augmented matrix containing the file M andthe coefficients used to code C . Initially, C is initialized to the identity matrix of the rightdimension. Let us call the augmented matrix beM ′ = [MC].

2. Generate a secret random vector v and compute the product u←M ′ ∗ vT , where ∗ denotesthe usual product among matrices and T is the transpose operator.

3. The client stores u (which is significantly smaller than the file) and sends linear combina-tions of the rows ofM ′ to the storage nodes.


. . .

P1

P|PN |

N . . .

C1

C|CN |

a)

. . .

P1

P|PN |

N . . .

C1

C|CN |

b)

. . .

P1

P|PN |

N . . .

C1

C|CN |

c)

. . .

P1

P|PN |

N . . .

C1

C|CN |

d)

EP1

EP|PN |

E′PN

E′PN

E′PN

E′PN

σ′P1

σ′P|PN |

σ′PN

σ′PN

σ′PN

σ′PN

Figure 4.1: Overview of the proposal: a) Each node Pi forwards a packet; b) N codes over them andbroadcasts; c) each Pi approves by generating a share of aggregate MAC or threshold signature;d) N combines the signatures shares and outputs the final threshold signature or aggregate MAC.

4. To perform a test over encoded information, the client sends information to the storagenodes so they can generate v. Servers send the vector of coefficients C and the result ofcomputing the function w over their stored files. The servers pass the test if u = w.

The actual construction stores everything at the server. For this reason, additional steps areneeded to hide u from the server. Security for the protocol is based on the fact that the productof a file with an unknown vector is unpredictable. Therefore, if the server loses the file, it cannotcompute the right value in advance. This gives the guarantee that the file was at the server whenthe protocol was executed.

If more tests are necessary, more secret vectors are computed before uploading the file. It isalso possible to design protocols that have an unlimited number of invocations such as [22], butthey are less efficient in terms of transmission overhead.

4.5 Proposed Solution to Prevent Diversity Attacks in Multicast Net-works

Taking advantage of the wireless medium, we can reduce the overhead in checking a signaturefor each member of PN compared to [7], by allowing each sender to check if their own packetwas included; then, we notify the verifiers using a single threshold signature [89]. Even thoughwe will use threshold signatures [89] for the description, aggregate MACs [96] can be used ina straightforward way. The proposal has a preliminary initialization step, after this, detectingwhether a node is performing a diversity attack involves the four steps shown in Fig. 4.1. There,Fig. 4.1.a represents the parent forwarding step where parents forward their information to N. Fig.4.1.b represents the packet sent by N to all the neighborhood. Fig. 4.1.c the generation of theparent approval message. Finally, Fig. 4.1.d processing at the children.

A more detailed version of the packets is presented in Fig. 4.2 and Fig. 4.3. In Fig. 4.2,packets are sent from the parents to node N. Then in Fig. 4.3 is is shown what parts of the packetare forwarded exactly. In particular, the information stating that a packet was sent from a parent

4.5. PROPOSEDSOLUTIONTO PREVENTDIVERSITYATTACKS INMULTICASTNETWORKS89

1

2 N C

3

1 1 1

2 2 2

3 3 3

# # #

This means packet sent by node “#”

1

3

2

Coefficients - 32 bytes

Payload - 1428 bytes

Pollution signature – 40 bytes

“From parent to N” - 40 Bytes

Packet sections

Figure 4.2: Packets sent from parents to N

1

2 N C

3

N N N 1 1 2 2 3 3

We use the fact that in Network Coding all

nodes can receive the packet

N

Overhead = 72*3 = 216 bytes

1 2 3

Compared to the PIP protocol, we do not transmit

this part

Figure 4.3: Packet sent from N to the neighborhood

to N is not transmitted. The intuition for this, is that parents know what packets they have sent;hence, this part is not needed if the packets will be checked by the parents themselves. This alsobrings savings in processing because the signature need not be verified by the parents. Fig. 4.3also shows how much information is saved with respect to the PIP protocol explained in Section4.4.1. This figure can be compared directly with Fig. 1.9 which has the same size parameters foreach part of the packet in the PIP protocol.


S creates for each set of nodes Pi, a (k, l)-threshold signature key pair (PkN, SkN). PkN is thepublic part of the key, that will be given to CN; one share of SkN is given to each one of themembers of PN. Also, all nodes in the network receive a public key that allows them to checkpollution signatures created by S. To transmit information the source creates packets of the formEPj = (Mj , Ej) where Ej is the j-th unit vector; then, the source signs each packet using apollution signature scheme to create σS(EPj); finally, packets are transmitted along with theirpollution signatures.

4.5.2 Parent Forwarding

Each node Pj ∈ PN forwards packetsEPj , their pollution signatures σS(EPj), and τ = σPj(αj ,σS(EPj),N).The pollution signature protects the packet from pollution and τ guarantees the message was infact generated by Pj.


Algorithm 7 Payload independent verification function at Pj1: check (αj , Cj ,σS(EPj),N) is present.2: check coefficient αi = 0.3: check EPN is not polluted by verifying σS(EPN).4: check if σS(EPN) =

∑

(αi · σS(EPi)).

4.5.3 Processing by N

When a packet is received, N verifies τ is a valid signature on the set of received packets; thenthe packet is checked for pollution by verifying σS(EPj) matches the packet. To perform networkcoding, N outputs a random linear combination of all the incoming packets by computing:

EPN =

⎛

⎝∑

j

αj ·Mj ,∑

j

αj · Cj

⎞

⎠ (4.8)

σS(EPN) =∑

j

αj · σS(EPj) (4.9)

after this, it appends τ = (αj , Cj ,σS(EPj),N) for each parent, to create the resulting packet

E′PN = (EPN ||σS(EPN)||τ1|| . . . ||τ|PN|)

that will be broadcasted. Note that unlike the original token τ used in the PIP construction, ourtoken is not authenticated; for this reason, our protocol incurs in less transmission overhead. Wewill prove in Section 4.5.7, why in our protocol this is secure.

4.5.4 Parent Approval Generation

When E′PN is overheard by parent nodes, they run Algorithm 7, which checks that a pollution sig-

nature that has previously been sent by Pj, was used to create the current packet. Since the packetwas already processed by node Pj, the only thing that needs to be done, is to check the packet isnot polluted, and that one of the signatures used to create the current signature, corresponds topollution signature σS(EPj).

After overhearing packets for a period of time (e.g. one generation), if Pj has overheard enoughpackets containing her previously transmitted packets; It uses its share of SkN, to sign its approvalmessage for the current time period σ′

Pj . This message is transmitted to N, which will combine theshares and forward them to CN.

4.5.5 Processing at the Children

When E′PN arrives, each Cj checks it for pollution and that it actually came from N, then the

packets are forwarded. To check for diversity, we must wait for the signature computed as acombination of the shares σ′

Pj ; this combination is computed by N once enough members of PNapprove N’s behavior. If a valid signature is not received within a given time period, N is labelledas misbehaving.


Table 4.2: Total overhead for N

Method OverheadNaive (|EPj |+ |τ |) · |PN|PIP |τ | · |PN|

Log-PIP |h| + 2 · |σS(EPj)| · log (PN) + |q|Proposal(N) (|τ |− |σPj(τ)|+ |σ′

Pj |) · |PN|

Table 4.3: Processing time

Method Processing TimePIP(N) (νS(EPj) + νPj(EPj) + χ) · |PN|PIP(CN) (νPj(EPj)|+ χ) · |PN|+ νS(EPN)PIP(PN) σPj(EPj)

Log-PIP(N) PIP(N) + 2 · (h+ χ) · |PN|Log-PIP(CN) νPj(EPj) + 2 · (h+ χ) · log (|PN|) + νS(EPN)Log-PIP(PN) σPj(EPj)Proposal(N) PIP(N) + χσ′ · |PN|Proposal(CN) νS(EPN) + ν ′PjProposal(PN) τ + χ · |PN|+ νS(EPN) + σ′

Pj

4.5.6 Effect of Replacing the Threshold Signature

Changing the types of digital signature used for the PIP construction and ours, has a significanteffect in the transmission overhead and execution times of the algorithm.

The different digital signature schemes that were described in Section 4.3.1 can be used indifferent situations. Batch verification for RSA [91] could be used in both constructions as anauthentication signature to reduce verification times. However, since all the signatures from PNare transmitted, this construction does not reduce the transmission overhead. Condensed RSA [92]could be used to reduce the overhead, but it still does not solve the problem that only signaturesfrom the same signer can be aggregated. On the other side of the transmission overhead spectrumare aggregate signatures. This kind of scheme transmits a single signature regardless of the numberof children |CN|. However, as shown in Section 4.3.1, signatures of this kind usually involve manyinvocations to a bilinear map function, producing a trade-off between space and complexity forverification. Another alternative regarding processing Aggregate MACs described in Section 4.3.2.MACs are very efficient to compute since they only involve a fast pseudo random function, butthey are not universally verifiable. For this reason, it is necessary to compute an aggregate MACfor each member of CN.

For all the schemes where signatures are somehow combined, it is also important to take intoconsideration the computational burden induced by the combination operation. For the case ofAggregate MAC the operation is the exclusive or operation which is very efficient; for the RSA-based schemes, the operation is a modular multiplication over big integers. For the schemes basedon bilinear maps, the combination operation consists in adding points from an elliptic curve. Wesummarize the number of operation performed at the verifiers for several types of digital signaturesin Table 4.4.


Table 4.4: Different types of overhead associated to different types of signatures at children nodes

Type of Signature No. parent signers (Verifications, Combinations) Space OverheadBatch Signature |PN| (1, |PN|− 1) |PN|

Threshold signature At least t (1, 0) 1Aggregate signatures |PN| (|PN|, |PN|− 1) 1Aggregate MAC |PN| (|PN|, |PN|− 1) |CN|

Wewill now provide some insight on what can be obtained by switching the signature schemesfor each of the schemes.

Aggregate Signatures on the PIP protocol

Consider replacing the authentication signature for the aggregate signature scheme from [93] ex-plained in Section 4.3.1. In the current PIP construction, a message from N to CN guaranteeingdiversity from PN, must transmit |PN| digital signatures. Using digital signatures with a length of20 bytes, such as the one proposed in [98], means that using an aggregate signature can reducethe transmission overhead from N to CN considerably. For instance, in a network with 5 children,we could avoid sending 80 bytes. Furthermore, considering that 32 is a popular value for thegeneration size and that field with 256 elements is popular choice for operations, this amounts toapproximately 27% of the total information transmitted from each parent.

The previous estimate follows from the fact, that in PIP, the information sent from each parentthat is not the authentication signature is given by:

∆ = (αi, Ci,σS(EPi),N) (4.10)

where αi is a coefficient (1 byte) used to multiply the vector from Pi; Ci is the set of coefficientsfrom that parent (32 bytes); σS(EPi) is the pollution signature which can be instantiated using 20bytes using schemes such as [57] for a small set of parents; N is an identifier (1 byte).

In terms of verification, the aggregate scheme from [99] has almost the same number of oper-ations as [98], making the proposal attractive for schemes already using bilinear maps. However,for constrained hardware, these signatures are more computationally intensive than other types ofsignatures such as DSA [87].

Aggregate Signatures on our construction

Our construction does not benefit from using aggregate signatures in terms of overhead. Thereason is that our construction does not need to send an authentication signature per parent. Infact, its main goal is precisely to get rid of the signature entirely. However, in terms of securityguarantees, a threshold signature is only telling us that at least t parents were satisfied with N,while an aggregate signature would tell us exactly what parents signed. This is important, since itpreserves the individual opinions from the parents in the aggregate signature. It can also be usedto have a greater granularity to identify misbehavior from N.


4.5.7 Security Evaluation

The main difference between our constructions and those from [7] is that node N does not forwardauthentication signatures from PN. We will now prove that omitting the signature from PN doesnot affect the security of our proposal.

Theorem 5. An adversary cannot pass the verification test of Algorithm 2 by using a different setof vectors from those transmitted in the packet EPN.

Proof. Consider an adversary who is able to create a valid packet E∗PN which passes the verifi-cation test from algorithm 7 for one node Pj. This implies, that node N was able to succeed in atleast one of the following scenarios:

1. N is able to produce pollution signatures on a different set of basis vectors, from those signedby S.

2. N is able to produce a pollution signature for a packet EPj , using a set of vectors signed byS that is linearly independent from EPj .

3. N is able to produce a pollution signature for a packet EPj , using a set of vectors signed byS that is linearly dependent from EPj .

The first scenario is not possible, since it contradicts the security definition of a pollution sig-nature, as stated in [30]. The second case is not possible, since the signature operates in boththe payload and coded vectors, and they are linearly independent by definition; to create such aforgery, we would need to create a signature outside the signed basis (from the adversary’s perspec-tive), which reduces to scenario 1. The third scenario arises when the set of packets transmittedby PN is linearly dependent; in such case, the transmitted packet contains the information fromEPj , so in this case the packet is transmitted, therefore no successful attack has taken place. Sincea vector representation is unique for a given basis, it is not possible to create a signature withoutforging the pollution signature.

When the proposal is instantiated using a threshold signature, it is secure as long as the numberof compromised users remains below the threshold. When nodes are compromised, these keys canbe learned by the attackers and used by N to create fake approval messages for herself. Hence ourscheme is t-collusion resistant for a given set of nodes PN. When i nodes have been compromised,we can only be sure that the coded packet contains information from t− i nodes, since the attackercan create the approval messages without forwarding any packets. For the case of the aggregateMAC, the attacker can still compromise nodes and learn their keys; however, this only allows anattacker to impersonate that particular node, and not any node as in the threshold case. A similarattack is possible in Popa’s et al. construction [7], since collusion among N and PN would allowthem to create signatures from a compromised parent on a packet that yields no new information.

Given that the information flow for approval goes from PN to CN, we need to take into accountmisbehaviors that might originate in PN. Given this, one security concern would be, a node Pj ∈ PNnot signing the approval messages on purpose; this would make nodes in CN believe N is not for-warding properly. In this case, N can show it is innocent, by forwarding τ = σPj(αj ,σS(EPj),N)upon request. The case where Pj does not forward any packet, is outside the scope of this problem.


Another related attack, would be a node Pj, generating a wrong version of its signature, to preventa correct signature combination at N; however, this attack is easily detected since all packets areauthenticated; hence, if the output of the combination operation cannot be verified by CN, thennode N simply needs to forward the authenticated shares to S who can immediately identify theattacker. In the case of aggregate MACs, authenticated signature shares can be sent to CN.

4.5.8 Probabilistic Alternative to Reduce Overhead

By carefully analyzing the previous protocol, there are some things that can be optimized basedon the following observations:

1. Honest nodes Pj send linearly independent vectors to N; this is meant to prevent redundantinformation. This must be true for the first g packets; after this, what should be checked isthat any subset of g packets are linearly independent.

2. Every time N receives a packet from Pj, the subspace that can be generated by N thanks tothe information obtained from Pj at a given time t expands. We call this subspace S(N, Pi, t).

3. The basis vectors sent from Pj are the base of S(N, Pj, t). Therefore, a vector belonging toS(N, Pj, t) is enough to know what packets sent by Pj have been received by N.

In other words, what the observations are telling us, is that by knowing a representative ofS(N, Pj, t), we can know what packets from a particular node were used for the current packet.Because of this, it is possible for Pj to check how S(N, Pj, t) expands by checking packets from N.

Recall from the protocol description that packets sent by parents contain information EPj =

(Mj , Cj), whereMj is the payload andCj are the coefficients vector for that given packet. ThenNcreates E′

PN = (EPN ||σS(EPN)||αj ||Cj ||N), which is the combination of the packets coming fromthe parents Pj, along with the coefficients vector from each parent Cj , as well as the coefficientused in the linear combination for that particular packet αj .

The first part of the probabilistic scheme, involves hashing the coding information (Cj ,αj)

for all nodes, using a collision resistant hash function H . Regardless of the number of neighborsonly a single value (around 160 bits) is appended to the packet. Since the output of H does notprovide enough information to verify N’s behaviour, random packets from a given generation mustbe selected for verification. For this step, the source selects what packets will be inspected for eachone of the nodes in the network. The procedure to perform the selection is as follows:

1. At the end of a generation, each node sends to its neighbors the list of hashes L of thepackets that were transmitted by it during that generation. The purpose of L is to have anunambiguous ordering of the packets sent by a given node. For a given node N, L would becomputed as follows:

ω(i) = (α(i)1 ,M (i)

1 , . . . ,α(i)|PN|,M

(i)|PN|) (4.11)

L = H(ω(1))|| . . . ||H(ω(t)) (4.12)

where ω(i) was the information used from each parent at a given time i.

4.6. PROPOSEDSOLUTIONTO PREVENTDIVERSITYATTACKS IN STORAGESYSTEMS95

2. Next, the source sends a special packet containing a random number x. Nodes in the networkuse an entropy extractor f (e.g. HMAC), which is a function that can be used to create acryptographically secure number from x. Function f is invoked as k = f(id, x), where idis the identifier of the node we wish to check.

3. k is used as the seed to a pseudo-random number generator (PRNG) to select the packetsuniformly. The order of the packets corresponds to the one sent by each node on the firststep.

Once the packets to be verified have been selected, N will forward the complete informationas in the usual protocol. In addition to the rules in the original protocol to generate the approvalmessages. Approval messages must not be created whenever any of the following situations occur:

• Hashes of the packets that were received and authenticated from N, do not form a subse-quence with list L. This means the node sent packets that were not reported in the list L,which constitutes inconsistent behaviour.

• List L is too long. This measure wishes to avoid an attacker that creates a very long list,where some messages were not transmitted. The purpose is to reduce the detection proba-bility for bad messages that were actually transmitted.

• dim(S(N, Pj, t’))− dim((S(N, Pj, t)) ≤ ϵ for t’ > t and ϵ > 0, where “dim” denotes dimen-sion of the subspace. This means N is not including enough new vectors from a parent inher packets.

Comparing our probabilistic construction with the probabilistic construction from [7], the dif-ference is: Log-PIP checks the contributions of random parents in all packets, whereas our prob-abilistic construction checks all parents in random packets. In terms of computational load, ver-ification and creation of our commitment scheme are more efficient because they do not use aMerkle Tree, but a single invocation of the hash function. In addition, the selection procedure isexplicit and more efficient in terms of bandwidth to authenticate a single packet. The selection ofϵ is an important parameter, since it indicates the frequency at which new packets from a parentwere included.

4.6 Proposed Solution to Prevent Diversity Attacks in Storage Sys-tems

The entire process consists of three routines: initialization, node regeneration and verification.This summarizes the scenario depicted in Section 4.2, where a source S distributes a file to a set ofstorage nodes (N1, . . . ,N|N|); we want to guarantee that all nodes Nj participating in the creationof a new node Cj, are transmitting a linear combination of all their stored data units.



The parent Pi has an initial set of data unitsM = (m1, . . . ,m|M |); next, Pi assigns to each storagenode (N1, . . . ,N|N|), λ elements of the form:

DNk,l =

|M |∑

j=1

αj ·mj , (4.13)

where 1 ≤ k ≤ |N|, 1 ≤ l ≤ λ. Each αj is a random coefficient.

To enforce the good behaviour of storage nodes at a later time, during the process of storingthis file or before deleting the local file, Pi will compute a number of PDPs Φ = (φ1, . . . ,φ|Φ|)

with linear properties as described in section 4.4.2 and store their values locally. These valueswill be used at the node regeneration stage, to check surviving storage nodes are transmitting validinformation to the new nodes. For security reasons, each φi value can only be used once; therefore,the number of PDPs to be computed depends on the expected number of node regenerations andthe time information needs to be stored.

4.6.2 Node Regeneration

To replace a failed one, a set of nodes Ξ ∈ (N1, . . . ,N|N|) sends β linear combinations of thefollowing form to the new node Cj:

ak,l =λ∑

j=1

αj ·DNk,j, (4.14)

for 1 ≤ l ≤ β and 1 ≤ k ≤ |Ξ|. Using the received data unitsA = (a1,1, . . . , a1,β, . . . , a|Ξ|,1, . . . , a|Ξ|,β) =

(a1, . . . , a|A|), the new node Cj will store λ random linear combinations of data units in A:

DCj,l =

|A|∑

j=1

αj · aj , (4.15)

for 1 ≤ l ≤ λ.

The condition for this procedure to produce a useful node Cj is explained in detail in [9].The intuition behind this kind of code is that there is a trade-off between parameters β which isassociated to the repair bandwidth (amount of information needed to repair a failed node) and λ(amount of information stored at each node). This trade-off can be used to repair failed nodesmore efficiently than traditional codes for storage such as Reed-Solomon [10].

4.6.3 Verification

There are two security concerns during the verification phase. The first of them is that Cj mustverify the information sent by nodes in Ξ was in fact given to them by Pi; the second is that thecombination of data units stored at Cj must be a valid linear combination.

4.7. PROPOSED SOLUTION TO DECODABILITY ATTACKS 97

Taking advantage of the fact that there is a direct connection between Pi and Cj (unlike the mul-ticast scenario), we can use a more efficient procedure that does not require a pollution signature,but the properties of the PDP.

1. Cj receives data units Ai = (ai,1, . . . , a1,β) from a storage node Ni; it also receives from Pithe information needed to compute an unused PDP φk and its result. Then, it must verifythat values Ai pass the PDP. This is done as explained in Section 4.4.2; the difference is thathere Cj is both the prover and the verifier. If data units Ai pass the test, they are used in thenode regeneration routine and discarded otherwise; this process is repeated for all the nodesin Ξ. Cj can compute the correct result for Ai because of the linear properties of the PDP.

2. Cj carries out the regeneration routine, by performing linear combinations of all the receiveddata units that passed the PDP protocol.

3. The final step involves Pi checking the behavior of Cj, by running the PDP protocol againstCj. This verifies that Cj is storing a correct encoded version of the file. In this stage Pi actsas the verifier and Cj as the prover.

The linear PDPs have two purposes during the execution of the verification routine. At first, thePDP is used to check information coming from other storage nodes is not polluted; whereas after,it is used to verify that the new storage node is storing a particular linear combination. After thisprotocol, Pi which in our scenario is the file owner, can verify the correctness of the regenerationoperation without receiving the information. This is important for cloud storage systems becauseit reduces the repair bandwidth.

In terms of computational load, unlike our solution for multicast systems which required apollution signature, this routine can scale better for large files.

4.7 Proposed Solution to Decodability Attacks

Decodability Attacks are related to Diversity Attacks in the sense that unlike pollution and entropyattacks, they also depend on the topology. In this section we will present a solution to DecodabilityAttacks based on the strategy used by the PIP protocol [7].

The intuition for the method to detect Decodability Attacks is that by having access to thestate of a given node when performing a coding decision, it is possible to verify that the node isbehaving correctly by repeating the computation at the receivers. In particular for this problem,we need to provide the following information to the nodes verifying the coder N’s behavior:

1. What packets have currently been received by N in the current generation. We call thisvector R and denote by Ri whether the i-th packet has already been received, we call thisvector the Received State Vector.

2. What packets are wanted by each one of the nodes CN in the neighborhood. There are |CN|vectors Wi, . . . ,W|CN|) whereWi = wi,1, . . . , wi,|Wi| denoting whether a packet is wantedby a member CN. Each member ofW and R can be represented as binary vectors keepinga consistent convention about the meaning of zeroes and ones contained in them.


Algorithm 8 Packet creation at N to prevent decodability attacksRequire: R: Packets received by N,W : Packets needed by members of CNEnsure: EpN: Packet to be transmitted1: C ← A(R,W )2: M ← 03: for each messageMj ∈ C do4: M ←M ⊕Mj

5: end for6: σS(EPN)← CreatePollutionSignature(M,C)7: return (M ||C||σS(EPN)||R||W )

3. Packets that have been XORed together to form the current packet. Following the conven-tion so far, we denote this vector as C , since it is equivalent to the coefficients used forpackets in linear network coding.

4. Payload of the packet represented by vectorM .

5. The coding strategy used by the network. A strategy is a public parameter of the systemtaking the form of a function A(R,W ) that given the current state of a node, outputs avector C that determines what packets in possession of a node should be XORed together.Some strategies are mentioned in Section 2.3. An example of a coding strategy could be tosend the XOR of at most two packets that allows the greatest number of members of CN todecode immediately.

Taking into account the previous definitions, we define the format of the packet to detectdecodability attacks from N as follows:

EpN = (M ||C||σS(EPN)||R||W ). (4.16)

In addition toM , C ,R,W , the packet also contains a pollution signature σS(EPN) that guaranteesis the result of applying the XOR operation over original packets according to parameters M andC . One instantiation of this kind of signature for XOR Network coding was presented in Section2.6.

To create a packet, N computes C ← A(R,W ) and proceeds to code the packetsM ←⊕

j∈C Mj ,hereMj represents the j-th packet of the current Network Coding generation. After this, N com-putes the appropriate σS(EPN) for the current values M,C and sends EpN to CN. Algorithm 8summarizes this process.

The verification procedure at a node Ci for a packet EpN consists of several steps. The firstof them is to verify that σS(EPN) is a valid signature over EpN .M and EpN .C . If the signatureis not valid, the packet should be discarded immediately. For packets with a valid signature, Civerifies whetherEpN .C

?= A(EpN .R,EpN .W ). If the values are equal, then N is behaving honestly;

otherwise, it is performing a decodability attack. Algorithm 9 summarizes the process.The definition of decodability attacks states that an adversary uses a coding strategy to min-

imize throughput. By making the coding strategy depend on the current state of a node, we areverifying that the node is behaving according to the intended protocol. Therefore, the protocolguarantees that given R,W and attacker cannot create a different packet from that suggested by


Algorithm 9 Packet verification at Ci to prevent decodability attacksRequire: EpN: packet to be analyzedEnsure: V : boolean stating whether node N is misbehaving1: V ← TRUE2: C ′, R′,W ′,M ′ ← EpN .C,EpN .R,EpN .W,EpN .M3: C ← A(R′,W ′)4: if VerifyPollutionSignature(EpN .σS(EPN), C

′,M ′) is FALSE ∨ C = C ′ then5: V ← FALSE6: end if7: return V

the coding strategy. Despite that, it is still possible for an attacker to send vectors R,W that arefalse. Now we present some strategies to deal with this problem:

ProtectingW : Nodes could authenticate their ownW vectors every time a packet is sent, thenN would forward the authenticators along with the other information on the packet. This issimilar to the approach taken by the PIP protocol from [7] and that is studied in Section 4.4.1.This kind of solution is receiver-centered because receivers must verify all the incomingauthenticators. A different approach is the sender-centered approach we proposed in Section4.5, where each node overhears packet for a given period of time and then approves ordisapproves the behavior of another node.

Protecting R: Protecting R is a problem that requires a different approach. The reason is thatin principle, only N knows what packets have been received by itself. This rules out thepossibility of using a receiver centered approach. However, by adding an additional criteriato the generation of the approval signature in the sender-centered approach, it is possible toprevent that N manipulates the advertised Rmaliciously. Candidate criteria that can be usedto monitor the malicious manipulation of the vector R from a node N include: increasednumber of received packets in vector R through time, number of useful packets receivedfrom N, ratio of received packets from N (useful or not) over number of packets sent thathave been reported by N.


Tests were performed in C++ in an Intel Core 2 Quad 2.66 GHz Q6700 running Ubuntu 11.10;all simulation values for times presented, are the average of 10000 runs. Our implementationfor the multicast scenario considers only overhead introduced by our proposal, and not the oneintroduced by pollution signatures, the reasons for this treatment are various: first, all packetsmust be checked for pollution in a secure network coding scenario; second, all the schemes from[7] and our constructions only check for pollution once in every stage of the protocol. Due tosome discrepancies in the constructions, they are not directly comparable in several aspects; forinstance: Log-PIP is probabilistic, and it involves an additional message stating what Pj will betested for inclusion; this node must not be known in advance by N, which means that somehownodes in CN should agree in what leaf to test, or perform several independent tests, which increasesthe overhead; our simulations optimistically assume inclusion for only one node. Regarding our


0

5

10

15

20

25

30

35

40

45

50

0 5 10 15 20 25 30 35

Tim

e (m

s)Number of Parents

PIPProp. MACProp. Sign

Log-PIP

Figure 4.4: Total processing time for a network with 4 children.

proposal, it does not need to send the approval message after every packet; however, we includedthe approval overhead in every packet to make the comparison fair.

Results for the simulation of the different schemes are summarized in Figs. 4.4 and 4.5. Bothfigures compare the payload independent schemes from [7] and our constructions for a scenariowith 4 children. Regarding the cryptographic primitives, we used 320-bit DSA [87] signatures asauthentication signatures for all schemes; to authenticate the approval messages for our MessageAuthentication Code (MAC) construction, we used the exclusive-or of the output of HMAC [63]using SHA1 as a hash function; for the threshold signature, we used 1024-bit Mediated RSA [89].Frequency for approval messages was set to 1 in every 32 packets, which is a reasonable sizefor a generation of network coding. Since we assumed a network protocol where all nodes wereperforming network coding, the actual values in the vertical axis, indicate the cost included by anode after performing its role as a parent, coder and child.

Fig. 4.4 contains the execution times; results show that our proposal has similar executiontimes to the Log-PIP construction. This is due to the savings in processing time, as a result ofprocessing only one message at each parent node, instead of performing all the checks at a childnode.

Fig. 4.5 shows the overhead incurred by the proposal. In this matter, the dominating factor isnot the approval signature, but the token containing the pollution signature for each of the parentnodes and the coefficient. For this reason, the overhead for Log-PIP is significantly lower, becauseit is only included for one parent. The difference between our proposal and the PIP constructionrelies on N not retransmitting the DSA signature to CN, which is significant once |PN| increases.Our probabilistic construction, has the same overhead as Log-PIP, but the actual overhead for onetest depends on the size of the neighborhood.

The advantage of using a threshold signature is, receivers can verify without any concernabout the number of signers (number or identities); thus, their processing overhead remains con-stant. The same reasoning applies to signers, who can sign regardless of the number of receivers;this makes the system secure and topology independent, as long as the collusion bound is not met.On the other hand, for each receiver in an Aggregate-MAC scheme, unless a smart key assign-ment is used, each receiver must generate one MAC for each sender; this impacts processing andoverhead in a linear way. In addition, it requires nodes in PN to know nodes in CN. Regardingour probabilistic scheme, the overhead grows linearly with the number of packets tested during


0

500

1000

1500

2000

2500

3000

3500

4000

4500

0 10 20 30 40 50 60 70

Over

hea

d (

byte

s)Number of Parents

PIPProp. MACProp. Sign

Log-PIP

Figure 4.5: Total overhead for a network with 4 children.

a generation. For each packet tested, the overhead is the same as that of a single packet in ournon-probabilistic construction.

To evaluate the feasibility of using aggregate signatures in the PIP scheme and our construc-tion, we implemented the aggregate signature scheme from [99]. We used the Pairing-BasedCryptography Library [100] for elliptic curve operations. To instantiate the elliptic curve, we usedone of the form y2 = x3 + x over Fq with

q =87807107996633125224377819847540498158068831994142082110286533992

66475630880222957078625179422662221423155858769582317459277713367

317481324925129998224791. (4.17)

This parameter was selected, since according to the library’s own terminology this is consideredas a Type A Pairing [101] which is the one with best performance. This parameter creates assignature a point on the elliptic curve, each coordinate has a length 512 bits. Our implementationtakes in average 35.5 ms to generate a signature, 39.2 ms for verification. In the aggregate case,we performed a simulation aggregating 10 different signatures into a single one, this procedureconsisting of adding th points on the elliptic curve takes 0.6ms. Verifying the aggregate signatureagainst the individual signatures took 318.5 ms, from this total time 7.1 ms were spent in theparent computation. The remaining time was consumed computing the SHA-256 function. Theresults show that even though it is feasible to use this kind of signatures to reduce the overhead inthe PIP-Scheme, it is still too computationally intensive to be used for every packet. However, thisprimitive can be used to replace the threshold signature in our construction to certify N’s behavior.

Transmission overhead for the storage system is in the order of a secure cryptographic key,needed to generate the random vectors involved in the protocol; the same applies for the verifi-cation routine. This is negligible in comparison with the size of files stored in storage systems.Execution times for the linear PDP from 3.3.2 are in the order of 16.03MB per second in the sameenvironment, without any additional optimization.


4.9 Conclusions

We presented a construction for the diversity problem in network coding for multicast and datastorage scenarios. For the multicast scenario, our simulations showed the non-probabilistic pro-posal has better execution times than other existing non-probabilistic constructions and similarexecution times to the fastest existing construction. As an additional advantage, our constructionis the first to work without any knowledge of the topology, making it suitable for highly dynamicenvironments. In addition, we suggested a change of authentication signature for Popa et al.’s PIPconstruction [7] based on Aggregate Signature [93]. This change can improve their transmissionoverhead, but it involves additional computational overhead compared to their original scheme.Compared to our proposal, the PIP construction using aggregate signatures still incurs in moretransmission overhead. Despite this drawback, this type of signature can be used to enhance theinformation provided by our detection method, since they provide greater granularity on the viewof each parent node in the system.

We also presented a construction to detect decodability attacks in XOR Network Coding. Theconstruction is similar in nature to the techniques used to detect diversity attacks, with the differ-ence that we need to consider the network’s routing strategy as part of the solution. The solutioncan be instantiated using either the sender-centered or receiver-centered approach as well as thetechniques for signature compression from Section 4.5.6.

Regarding the construction for the data storage scenario, we proposed the use of a Proof ofData Possession to guarantee the regeneration stage was done successfully with a very efficientprotocol that can be executed by the data owner without having access to the new storage node’sinformation. This takes advantage of the fact that the information owner can contact the new nodeduring the regeneration routine.

Chapter 5

Aggregating Encoded Data

103

104 CHAPTER 5. AGGREGATING ENCODED DATA

5.1 Introduction

With the advent of cloud computing as an important tool used to store financial data, users areincreasingly dependent on the security of service providers to keep their information secure; how-ever achieving this goal in practice is really hard. Even if information is secure from externalattackers, cloud service providers must comply with government regulations in terms of supplyinguser data, in cases such as a criminal investigation (see [102] for a sample customer agreementclause); another security concern arises when insiders with enough access privileges, are able tobypass the security controls protecting that information. In order to prevent such attacks, in thischapter we will focus on the problem of storing and retrieving data securely from a cloud-basedexpense tracking application, where users can manage their receipts using an online interface. Thischapter will present our constructions to address the challenges presented in Section 1.5 for thisscenario involving encoded additions of receipts.

Managing receipts is a common task performed in businesses and homes alike worldwide,usually when some sort of good is acquired or a service is received, a receipt is given to the userdetailing all the relevant information of the transaction that just took place. If this information isneeded later for matters such as: refunds, warranties, accounting or tax related activities; users areendowed with the task of organizing, storing and retrieving this information themselves, which isa significant logistic problem when the number of transactions is significant.

In order to ease this task, several companies (e.g. [103], [104]) have developed different kindsof solutions; they could be categorized in two approaches: one where the user submits the actualreceipt (physically or electronically) for later processing by a company employee; and the otherwhere the physical receipt is replaced by sending a digital copy to a mobile device or server. Bothapproaches have their advantages and disadvantages, on one hand the human capture approachdoes not need cooperation at points of sale (POS), but is intrinsically vulnerable to an insideattacker; on the other hand, the digital receipt approach needs cooperation from the POS whichlimits its implementation. In both cases, a database or another means of storage provides onlineaccess to the information.

In this section we will present three efficient architectures for performing fast and secure ag-gregation. Two of them are based on additively homomorphic cryptosystems, while the third oneis based on a secret sharing scheme and requires no cryptographic key at all. Beyond the contextof this particular application, we propose a fast aggregation technique of theoretical interest inSection 5.5.3. Since one of the crucial goals of this work was to provide a practical implemen-tation, we provide an evaluation of the impact of adding cryptography to the system in practicalterms; which shows the feasibility of the proposal in a real scenario.

To perform operations in databases storing encrypted data, other techniques are of great usebesides homomorphic encryption, as it is presented in [105]. In particular, Order PreservingEncryption (OPE) [106], [107] and Efficiently Searchable Encryption [108], could be applied toprovide privacy and some efficient types of queries, for fields other than the actual price paid fora product. Even though the study of these primitives is an active topic of research within thecryptographic and database communities; in this chapter we will confine ourselves, to the onesrelevant to this particular type of application, without claiming any novelty about them. However,

5.2. PROBLEM SETTING 105

the exposition of existing work in this field, will serve as a way to prepare the scenario for theexperiments and point out the limitation of existing work when data wants to be aggregated.

The contribution of this chapter comes from several aspects, the first of them is that it is thefirst to the best of our knowledge, to address the specific problem of storing and aggregatingreceipts online. Second, we present a dynamic programming algorithm to aggregate informationefficiently, which can be used in databases encrypted with additively homomorphic cryptosystems;we present a proof of its correctness and security. Finally, in practical terms, our proposals canbe deployed without significant changes in user interaction at POS, in ways that benefit users andPOS alike, since the latter could use the proposal to outsource their own financial data.

The rest of the chapter is structured as follows: in Section 5.2, a definition of what wants tobe achieved in terms of security and ease of use of this system is presented; next in Section 5.4,three existing algorithms with with additive homomorphism are explained; then in Section 5.5, wepresent our proposals for an architecture of a receipt application from the cryptographic point ofview, and how the schemes can be applied to different scenarios; finally, in Section 5.6, we presentsome experimental results based on our implementation of both algorithms; the conclusions of thischapter are presented in Section 5.7.

5.2 Problem Setting

There is a set of users U = {u1, . . . , um} wishing to perform transactions to acquire goods orservices from a set of businesses B = {b1, . . . , bm}. When a transaction happens, ui ∈ U and bj ∈B engage in a protocol which generates a record dl in a database D, containing the informationof the transaction (receipts). The problem consists of guaranteeing the secrecy of the informationcontained in dl, while still allowing aggregation queries; privacy must be preserved, even when anadversary gains access to D.

Given the mixed nature of information contained in a receipt, such as: price of products,name of the store, dates, and others; we define secrecy for our application only for the price ofproducts, using the notion of security under chosen-plaintext attack (IND-CPA) represented bythe following game: given two plaintexts m0 and m1 and an encryption oracle which encryptsone of them randomly and returns the output (without telling which plaintext was encrypted), anadversary A cannot distinguish which of the plaintexts was encrypted, with an advantage greaterthan 1/2 + |ϵ| where ϵ is negligible.

The IND-CPA security notion captures a desirable property for security, which is somebodywho sees a ciphertext, cannot infer any information about the corresponding plaintext. Also, thisnotion of security is the strongest that can be satisfied by homomorphic algorithms. The reasonfor this can be explained by taking a look at other security games that provide more power tothe adversary. Security under chosen-ciphertext attack (IND-CCA) and stronger security notions,give the attacker not just the ability to call the encryption oracle, but also a decryption one. A

wins the game if she can decrypt a ciphertext that was not known to the oracles. This notion ofsecurity cannot be satisfied by homomorphic encryption, sinceA can create new ciphertexts whosecorreponding plaintexts are known. Consider a function E(k,m) such that E(k,m) ∗ E(k,m) =

E(k,m +m) for some operations (∗,+). To win the security game, A could ask the encryption


oracle to encrypt plaintextm , to produce c = E(k,m); given the homomorphic property, A knowsthe decryption of c′ = E(k,m) ∗ E(k,m) = E(k, 2m), which was not produced or decrypted byany of the two oracles.

To evaluate the feasibility of a given scheme, we will evaluate how practical it is using thefollowing criteria:

1. User actions: Number of steps performed by the user when a receipt must be generated.

2. User storage: Size of the information that must be stored by the user to participate in atransaction.

3. Computational load: Computational load at S, terminals where transactions occur and ter-minals belonging to users to surf their receipts online.

4. Hardware requirements: Requirements of hardware needed by users and businesses toparticipate in the system.

5.3 Existing Tools and Solutions

In this section we will present constructions to perform operations on encrypted data; the first twoconstructions that will be described allow equality queries and range queries respectively; the thirdone is a practical C++ implementation, of different kinds of techniques in the field of encrypteddatabases. As it is expected from the first two constructions, the notion of security achieved bythese schemes is weaker than IND-CPA. Despite this, primitives of this kind are important fromthe practical point of view, since they present a trade-off between security and usability for realproduction systems.

5.3.1 Searching on Encrypted Data

Within the database community there are some constructions that allow to search for equality ofencrypted texts. We will present the analysis from [108] to show how this primitive is; according totheir naming convention, the following scheme is called Encrypt-and-Hash Efficiently SearchableEncryption (ESE) :

1. Let E(Pk, x) be a public key encryption scheme, whose public key is Pk and operates onmessagem. LetD(Pk, Sk, y) be its decryption function, where Sk is the private key and yis the ciphertext.

2. To store x in the database, the client computes h ← h(Pk||x) and y ← E(Pk, x); y||h issent to the server. The server should index tag h for fast retrieval.

3. To query, the client recomputes the function and searches for h in the database.

4. To check the returned values are not false positives, the client computes:

• Obtain x← D(Pk, Sk, y).

• Compute h′ ← h(Pk||x).

5.3. EXISTING TOOLS AND SOLUTIONS 107

1 2︸︷︷︸

1

3 4 5︸︷︷︸

2

...︸︷︷︸

...

... N︸︷︷︸

M

Figure 5.1: Resulting map after applying the hypergeometric distribution to the domain and range[109]. The mappings do not overlap; therefore, preserving the order relations of the plaintext spacein the cyphertext space.

• The value is not a false positive if h′ = h.

The idea consists in storing a deterministic function of the plaintext in the form of a tag atthe server, without revealing any information about the plaintext in the process. Then the servercan process data as usual in sublinear time. The construction is secure as long as there is minimalentropy, otherwise an adversary with access to the database, would be able to match ciphertexts toplaintexts using exhaustive search on the tags using the available public keys.

5.3.2 Range Queries

One way to provide range queries over encrypted data, is to create encryption schemes that pre-serve the order relationship of the plaintext space in the ciphertext space.

In [106] Agrawal et al. propose a system called OPES which aims at solving the attacks facedby previous attempts. Before this work, the proposed solutions usually allowed an adversary toinfer the input distribution of the plaintext by looking at the ciphertext distribution. To solve thisproblem, they propose to map the ciphertext to a given target distribution; the actual procedureinvolves partitioning the data, flattening the distribution and then mapping the result to a targetdistribution. To decrypt the information, a state depicting the mapping operation is used to performthe reverse operation. The authors provide no formal analysis of their construction.

A provable approach is presented in [107] by Boldyreva et al. their construction is based on theHypergeometric Distribution (HGD). The HGD gives for an urn with N elements, the probabilityof having k successes in n draws without replacement, where there are M elements that can beconsidered a success. The encryption function is a map from

{1, . . . ,M}→ {1, . . . , N}, (5.1)

what the algorithm does is to split the domain in halves, similar to a binary search; for eachintermediate index y, HGD is called to determine how many values x from the domain shouldhave been sampled after y trials. The key is used as the randomness for the sampling procedure.The encrypted value for x, is a random value from the part of the range mapped to the value.

An example of a resulting map can be seen in Fig. 5.1, where elements inside the boxesrepresenting the range {1, . . . , N}, are mapped to elements of the domain represented by theunderbraces. For this particular example the encryption for the value 2 could be any value between3 and 5 inclusive. To decrypt, the function is recomputed again, when the size of the currentdomain after performing the binary search equals one, the member of that domain is the plaintextvalue. Continuing with the example, the decryption of any of the values {3, 4, 5} equals 2.


It is worth noting, that it is not necessary to preserve the order of the plaintexts, to performrange queries on encrypted data. For instance, in [109] Boldyreva et al. propose the use of mono-tone minimal perfect hash functions which are special types of hash functions, that preserve theorder of the plaintexts and have an almost linear storage requirement; to instantiate this kind offunction, they suggest the use of the functions presented in [110], where one has O(n log logw)

storage requirements and O(logw) evaluation time, and the other has O(n logw) storage require-ments and O(1) evaluation time; where n is the number of elements that belong to a universeof 2w elements. The idea of the construction is simple, yet powerful; it consists of taking theplaintexts mi and finding their positions in the set according to the hash function h(k1,mi); thishash function uses a key k1 that allows to change the mapping to the domain of the functionfor different users. Then, a secure encryption algorithm E(k2,mi) (e.g. AES) which dependson key k2, is used to encrypt the information; the information that is stored in the database ish(k1,mi)||E(k2,mi). When a range query for the range [a, b] with a ≤ b wants to be performed,the client issues a query for the range [h(k1, a), h(k1, b)]; since h(k1, a) ≤ h(k1, b) when a ≤ b

for all a, b in the domain, the database replies a simple range query as if the information was notencrypted at all. The authors show the scheme satisfies their requirements for security as long asE is IND-CPA. Even though this construction provides a stronger notion of security than the orderpreserving construction; it is hard to apply in our scenario, because the set of plaintexts must beknown in advance to instantiate the hash function efficiently.

At the time of the writing, we did not find in the literature, schemes with sub-linear searchtimes with traditional notions for security. As pointed in [109], the problem in satisfying strongsecurity notions, relies on the fact that even an ideal function that preserves the order of the plain-text domain, still reveals some information about distance of the plaintexts. For a linear searchtime scheme, that allows range queries on encrypted data in the public key scenario, we refer thereader to [111].

5.3.3 Research Related to Aggregate Queries

The problem of aggregating information in encrypted databases using homomorphic encryptioncryptosystems, has been treated in the literature before. Being the two more relevant [45] and[112]. The second scheme presents a public key homomorphic algorithm, which was shown to beinsecure in [45]. The first article, presents an approach similar to our proposal from Section 5.5.3,but whose execution times are slower than ours.

The scheme in [45] is based on bucketization, the intuition behind their system is to storethe result for a set of given values before hand, these results are encrypted using a homomorphicencryption algorithm. When the database is updated, the bucket is also updated, using the homo-morphic property of the algorithm. Consider Table 5.2, where one table represents the employeesand their salaries, while the other represents buckets to retrieve the information faster. If a queryasking the total budget spent in paying employees who earn less than 40000 is issued, the serverwill simply respond with the value corresponding to E(80000) which can be retrieved efficiently.However, if the query does not match the buckets perfectly, such as budget for employees whohave salaries lower than 30000 but greater than 40000, then all the rows from the bucket of Fig.

5.3. EXISTING TOOLS AND SOLUTIONS 109

Name SalaryA E(25000)B E(45000)C E(20000)D E(35000)

(a) Employees

Id Range Count1 [2000, 40000] E(80000)2 [4000, 60000] E(45000)

(b) Buckets table

Figure 5.2: Sample scenario for the bucket algorithm from [45].

Client:id=1 App. Server CryptDB DB

Get my posts posts(id=1)

[p1, . . . , pn]

posts(id=F (k1, 1))

[k1, p1, . . . , pn] [F (k1, p1), . . . , F (k1, pn)]

Figure 5.3: Example of a selection query in CryptDB for a forum application. For the applicationserver CryptDB looks like a MySQL or PostgreSQL database; queries and are parsed using theappropiate keys. Existing applications can work under this architecture, wihtout any modifications.

5.2 need to be sent for processing; the client issuing the query must decrypt to filter false positives;for the worst case, the entire database must be processed.

Our construction from Section 5.5.3, does not suffer from this drawback, and only requiresone decryption since it does not retrieve any false positives.

As an alternative to homomorphic key cryptosystems, linear secret sharing schemes have alsobeen used in the literature; we describe a scheme of this characteristics in Section 5.4.3. The sameconstruction is also presented in [24].

5.3.4 CryptDB

In [113] Popa et al. presented CryptDB which is a working implementation and open sourceproject, which implements exact query matching, range queries via Order Preserving Encryption[107], homomorphic operations on data using Paillier’s Cryptosystem (see next section for details);and linear complexity encrypted text search (SQL’s LIKE operator) using a protocol from [114].

The application uses MySQL proxy, as a means to intercept the database calls from the realapplication before they reach the database; in this way, they are able to translate the queries anduse the proper keys, to present the results to the server. To the eyes of the server CryptDB lookslike a normal database, The architecture is summarized in Fig. 5.3 where an example is presentedfor a selection query.

Performance tests done by the authors of the original article showed that the performancepenalty was 14.5 % for the phpBB forum application. This proves that CryptDB is a good optionfor applications that do not perform, aggregation queries. We will show using simulations, thatthe throughput of the systems is further reduced whenever aggregation queries are performedusing Paillier’s homomorphic cryptosystem. Therefore showing our different constructions are ofpractical relevance to implement encrypted databases in practice.


5.4 Algorithms with Additive Homomorphism

In this section, we present three secure algorithms with additive homomorphism: the first one issymmetric, the second one is asymmetric. and the final one does not require a key, but providessecurity by assuming the existence of several non-cooperative servers. We present their construc-tions to outline advantages and disadvantages of the key paradigm on which they are based.

5.4.1 Efficient Data Aggregation in WSNs

In [25] Castelluccia et al. present an algorithm for efficient aggregation of information in sensornetworks, this algorithm is symmetric, possesses an additive homomorphism and is shown to havesemantic security against chosen plaintext attacks (IND-CPA); its construction is as follows:

The setup phase of the algorithm involves these parameters:

fs(r) : a pseudo random function (PRF) depending on a secret parameter s (the key) and a randominput r.

h : uniformly distributed hash function, with the same output length as f .

M : an integer representing a modulo larger than the sum of all possible inputs.

The encryption function is c = m+ h(fs(r)) mod M where m is the plaintext and the out-put of h is treated as a number; to decrypt, the opposite operation is applied m = c − h(fs(r))

mod M . To aggregate a set of ciphertexts C = {c1, . . . , cn} created from plaintexts T =

{m1, . . . ,mn} into an aggregate a, we compute the following (all sums are performed moduloM ):

a =n∑

i=1

ci =n∑

i=1

[mi + h(fsi(ri))]

=n∑

i=1

mi +n∑

i=1

h(fsi(ri)) (5.2)

To decrypt the sum of plaintexts, all the pseudo random values are subtracted from a. It is im-portant to note that keys and random inputs used to aggregate ciphertexts need not be different,the sum of the original plaintexts can still be recovered by knowledge of the key and random in-put used to generate each corresponding ciphertext, which explains the additional index used forsecrets si in (5.2).

In the context of the original article, different nodes in a network add their own value usinga secret si particular to each node, this value is generated from a master secret K owned by thesink. Decryption is possible because si = g(K, i), where g is a pseudo random function, thusallowing the sink to reconstruct the keys used by nodes in the path, without giving nodes in thepath information about K . The security of the construction follows from the function f , whichproduces output that cannot be predicted by a bounded adversary without access to the keys.

Using the same principle, the authors also present an authentication function, based on a globalnetwork key k and a shared key among the sink and each node (ki); the construction is as follows

5.4. ALGORITHMSWITH ADDITIVE HOMOMORPHISM 111

(operations are moduloM ):

y =n∑

i=1

[(h(fki(ri)) +mi · h(fk(ri))] (5.3)

In order to check for the authenticity of data, the sink decrypts and finds the input parametersof (5.3); if the value computed by the sink matches the received value, it is considered authentic.This method for authenticity only works if k is not compromised, making it resilient to the fullcompromise of 1 node only.

5.4.2 Paillier Cryptosystem

In [23] Paillier proposed a public key additive homomorphic cryptosystem based on the hardnessof computing n-th residue classes; in order to describe the scheme we will use the traditionalnotation Zn for the additive group of integers modulo n and Z∗

n for the multiplicative group. Thescheme is as follows:

1. Select sufficiently large prime numbers p and q to compute n = pq, then compute λ =

lcm(p− 1, q − 1), where lcm is the least common multiple function. Now select a randomg coprime to n2, such that n divides the order of g in Z∗

n21.

2. The public key is: (n, g), and the private key is: (p, q).

3. The encryption function, takes a plaintext m ∈ Zn, and a random element r ∈ Z∗n, and

computesc = gmrn mod n2 (5.4)

4. To decrypt c we first compute x = cλ mod n2 and y = gλ mod n2, then the decryptionfunction is given by

m = L(x)L(y)−1 mod n (5.5)

where L(u) = u−1n .

The additive homomorphism is achieved by multiplying two ciphertexts modulo n2, which resultsin the encryption of the additon of the plaintexts, as the following algebraic manipulation shows

E(m1) · E(m2) = (gm1r1n)(gm2r2

n)

= gm1+m2(r1r2)n

= E(m1 +m2) (5.6)

The system is semantically secure against chosen ciphertext attack (IND-CPA). In order to provideintuition for this claim, consider the encryption function, it describes the following isomorphism:

Zn × Z∗n → Z∗

n2

(m, r) → gmrn mod n2 (5.7)1If x is the order of g in the group Z∗

n2 , then x is the smallest number in Z∗

n2 such that gx ≡ 1 mod n2.


as it is seen, the function consists of multiplying a number in Zn times an n-th residue modulo n2.The underlying problem that needs to be solved to decrypt is called the Computational CompositeResiduosity Class Problem, which is defined as: given a ciphertext c ∈ Z∗

n2 and the base g,compute the only plaintext m ∈ Zn that can produce such output. The semantic security comesfrom the fact that n different random values produce ciphertexts that decrypt to m, making theciphertexts uniformly distributed in Z∗

n2 .Regarding the efficiency, the encryption function performs two exponentiations and one mul-

tiplication modulo n2, while decryption consists only of a single exponentiation given that y canbe computed during the key generation stage.

5.4.3 Secure Addition Using Shamir’s Secret Sharing Scheme

In [44], Shamir introduced the notion of a threshold scheme, which is a way to share a secretamong n parties such that, at least k + 1 of them are needed in order to reconstruct the originalsecret; if less than k + 1 parties cooperate, they do not obtain any information about the secret.The construction of the system is based on polynomial interpolation in a finite field; its executiontime is very efficient in practice. The first step involves selecting a random degree-k polynomial,where the free term a0 will be the shared secret.

f(x) = akxk + ak−1x

k−1 + . . . + a1x+ a0 (5.8)

Then the polynomial is evaluated at different points i = 0 to form n secret shares of the form(i, f(i)); each one of the n parties receives a share. When the secret key wants to be reconstructed,the k + 1 parties provide their secret shares to the entity performing the reconstruction; usingthe shares, the polynomial is interpolated and the secret is recovered. Even though many waysto perform the interpolation exist in the literature, we will stick to the Lagrange interpolationpolynomial, since it can recover the free term directly.

Assume the shares available for reconstruction are of the form (xi, f(xi)), in case more thank + 1 shares are available, select k + 1 of them; this does not affect the result, because all theshares belong to the same polynomial. Now compute the free term by evaluating the followingformula at x = 0:

f(x) =k∑

j=0

f(xj) ·∏

0≤m≤km=j

x− xmxj − xm

(5.9)

The claim that any combination of k + 1 shares recover the same polynomial, follows fromthe fact that there is only one polyonmial of degree at most k, that goes through any of the k + 1

shares. The construction is secure because, we can construct polynomials sharing k shares withthe original one, whose free term can be any of the other elements in the field different from a0;hence, getting k shares or less, reveals no information about a0.

The previous formula also shows another important property of polynomials, which is: we canrepresent them using coefficients as in Equation 5.8 or as shares. In other words, to perform oper-ations on the polynomial, the results are the same no matter what representation we use. Considertwo polynomials f(x), g(x) that are represented using shares evaluated at common points; then,

5.4. ALGORITHMSWITH ADDITIVE HOMOMORPHISM 113

when we want to add these polynomials, we simply add (i, f(i)) + (i, g(i)) = (i, f(i) + g(i)),this is the same as performing the sum on the other representation. Multiplication is also possi-ble; which is a fact that has been extensively used to perform fast polynomial multiplication usingthe Discrete Fourier Transform (DFT); however, we cannot use this for our purposes, because thedegree of the polynomial changes when is multiplied by another one; hence, we would need toprovide more shares to each server, which in constrast reduces security.

If we combine the threshold properties of polynomial interpolation as shown by Shamir’sSecret Sharing Scheme, and the equivalence of representations; we can perform secure addition ofsecret information as follows:

1. The number that wants to be hidden (e.g. the price of an article) is set as the free term ofa k-degree polynomial fs(x), where k + 1 is the number of independent servers providingstorage. The other coefficients of the polynomial are assigned randomly.

2. Each server has an identification number starting from 1, that will remain constant through-out the lifetime of the system.

3. Each server receives fs(id) according to the id each server.

4. When an aggregation query is to be performed, the server adds the the shares correspondingto each of the articles fs(x).

5. The entity wishing to recover the secret (the user), interpolates the received polynomial andfinds the aggregate result.

The system is secure and requires no keys as long as the servers storing the information do notcollude. This idea was used in [24] by Thompson et al. where also a way to guarantee authenticityis provided. Their method consists in using Pedersen’s cryptographic commitment [115] and aMerkle Hash Tree [37]. First, Pedersen’s commitment is used; next, the result is used as the inputof the Merkle Hash Tree commitment; finally, the output of the last commitment is signed usinga digital signature algorithm. We will briefly sketch the scheme for the sake of completeness,but we refer the reader to the original paper for a detailed description. Commitments are usedin cryptographic protocols to force an adversary to use the same values throughout the wholeexecution. Pedersen’s commitment is based on modular arithmetic and has some homomorphicproperties; it receives as input the data m to be committed and a random value r. The purpose ofthe random value is to add randomness to the commitment, such that commitments with the samevalues look different. This is important in the context of databases, because an adversary cannotuse an entry several times to produce invalid results without being detected. The commitmentsfrom the previous step become the input to Merkle Tree; this is, put the hash of each commitmentas the leaf of a binary tree; each internal node of the tree is the hash of the concatenation of itschildren. Finally the root of the tree is signed using a digital signature. The server transmits thesum of the values, the sum of the random values for the commitments, the commitments and thesignature of the root of the hash tree.

To check that an aggregate query is correct, the signature on the root of the hash tree is verified;then, using the homomorphic properties of Pedersen’s commitment, an aggregate commitment is


created. If the output of the aggregate commitment matches the commitment created using thesum of the values and the sum of the random values, the query is declared authentic.

5.5 Proposal

In this section we will show the actual constructions to protect the privacy of outsourced financialinformation, even in the scenario where an authorized third party gains unauthorized access to thedatabase.

Using cryptography in a mainstream scenario introduces significant challenges related to us-ability and user interaction. In particular for our case, members of U must be able to hold acryptographic key in two parts of the process: the first, is to provide a key to members of B duringa transaction; the second, is when they wish to access their online information.

Given the characteristics of the cryptosystems and schemes presented in Section 5.4, we willdevise three different scenarios. In all of them, we will assume there is an effective authenticationprocess, and that the communication is secured using a protocol such as Transport Layer Security(TLS).

5.5.1 Scenario Using the Symmetric Scheme

The protocol for recording information in S has 3 steps:

1. ui ∈ U creates a session key kt, this key is derived from a publicly known key derivationfunction D(K, t), where t is an increasing parameter and K is a secret key only known toui.

2. A point of sale bj ∈ B, receives kt and computes the encryption of the prices ml given bycl = ml + h(fkt(l)), where the parameter l is set to be a counter representing how manytimes a particular key has been used.

3. The encrypted result and information needed for recovery {t, l} are sent to S.

ui bj S

{D(K, t) = ks} {cl = ml + h(fks(l)), t, l}

Figure 5.4: Steps performed at POS in the symmetric scenario.

When ui wishes to access its online information, it sends a query Q to S for an encryptedaggregated result a and the information indicating how to decrypt a properly, which will be de-noted by d; it is important to mention that d does not provide enough information to deduce kssince bj does not know K . Given that a single a can include many sets of values encrypted withdifferent keys, d must include all the necessary information about indices for the keys that mustbe used, how many times they must be used, and their respective starting counters. For instance,if a is the sum of 3 encrypted elements {c1, c2, c3} where the first two were encrypted with keykx starting at l = 5, and the last one was encrypted with key ky starting at l = 8; we will state

5.5. PROPOSAL 115

that the operations that need to be performed to recover the plaintext version of a are given by{(c1, x, 5), (c2, x, 6), (c3, y, 8)}.

Using the previous representation for the operations and the fact that we do not need to recoverthe plaintext for a particular ci, using d = {(x, 2, 5), (y, 1, 8)} suffices to decrypt a correctly, ifinterpreted as: kx should be used twice starting at counter 5 and ky should be used once startingat counter 8. In general we will define d =

⋃

i∈Q (αi,βi, γi), where each operation (αi,βi, γi)

means that the key whose index is α must be used for decryption β times starting at counter γ.

S ui

{a =!

i∈a ci, d ="

i∈Q (αi,βi, γi)}

Figure 5.5: Information transmitted to decrypt the aggregate result a.

Once ui obtains a and d, it is able to decrypt by subtracting the output of the values obtainedby the encryption routine, as shown in Section 5.4.1.

In terms of user actions, providing a session key for every transaction can be problematic.As an alternative, one key could be reused a small number of times (e.g. purchases of one day),which is secure as long as members of B do not collude. Certainly the logistics involved in keydistribution, are a significant drawback for users, but if a member of B wishes to have his ownprivate database on the cloud to which several terminals connect, this scheme is a good alternative,since it is not computationally intensive. Given the nature of this algorithm, one drawback inthis setting comes from communication overhead for cases where aggregated records returnedby a query, were not stored continuously in the database; hence, demanding the transmission ofadditional operation tuples for decryption.

In terms of user storage, in this scheme it consists solely of one symmetric key used to generatesession keys in every execution of the protocol. To implement this in a real scenario, the followingtwo options could be used:

1. Secure hardware: a smart card storing K could be used to generate ks every time it isneeded. This is fast and inexpensive from the user’s point of view at the encryption stage,but involves new hardware acquisition at participating stores; as an additional drawback,since decryption involves the use of different keys, users must acquire a smart card readerfor the decryption stage, which could become a bottleneck for the throughput of the de-cryption algorithm, in cases where the user has performed many transactions. However,the increasing availability of embedded smart card readers in consumer cellphones, coulddiminish the importance of this problem.

2. Generate keys beforehand: users could generate keys beforehand and have them in a printedform that could be accessed by POS terminals. This solves the problem of hardware acquisi-tion by B but involves more user interaction, making the system inconvenient for the users’perspective. The printed form could be replaced by a barcode on the cellphone screen, orany other short distance communication mechanism.


S bj

{Pui}

{E(Pui,mi}

(a) Store

ui S

{Q}

{a}

(b) Query

Figure 5.6: Diagram of the asymmetric scenario.

5.5.2 Scenario Using the Asymmetric Scheme

Every ui ∈ U generates a pair of keys {Pui,Kui}, where Pui is the public key and Kui is theprivate key; once this is done, ui registers Pui in S. When a transaction is going to take place, bjgets Pui from S and then encrypts the financial information mi to produce ci = E(Pui,mi).

The decryption routine is as follows: when ci wishes to access its online information, it sendsa query Q to S for an encrypted aggregated result a, that is decrypted by ui by computing m =

E−1(Kui, a).This scheme is straightforward to implement and involves no participation from ui, because

public keys can be stored on a server or in a barcode printed on a membership card, which iscompatible with existing infrastructure at participating stores. On the downside, this algorithm ismore computationally intensive for encryption than the symmetric alternative. In addition, thereis an overhead in time for the aggregation routine and additional storage due to the size of thenumbers in the ciphertext space.

Systems involving the use of homomorphic encryption for arithmetic operations, usually fol-low this paradigm. Cryptosystems such as Paillier, where the aggregation operation is a modularmultiplication of big numbers (around 2048 bits), will put a significant burden on systems per-forming extensive arithmetic operations. To solve this, we will now explain how to reduce theload at S when performing aggregation of rows, using a dynamic programming approach.

5.5.3 Fast Aggregation

The most computationally intensive activity performed at the server is the aggregation of informa-tion, which could be significant if the number of results is considerable.

First, we will introduce some notation used throughout this section: the i-th row in a table ofa database will be denoted by ri, to denote the j-th column of that row we will use ri,j where jis indexed from 0; an interval starting in row k (inclusive) and ending in row l (inclusive), willbe represented by [k..l]. In addition, for clarity purposes we will use multiplicative notation foroperations performed in the ciphertext space and additive notation for the plaintext one.

Now we present how a server can perform aggregations in a better way using a memoizationstrategy:

1. When a column in a database is meant to participate in aggregation queries, the user sendsE(mi).

5.5. PROPOSAL 117

2. For a row ri the server stores information as

ri =

⎧

⎨

⎩E(mi),

i∏

j=1

E(rj,0), r−1i,1

⎫

⎬

⎭(5.10)

It is important to mention that ri,1 can be computed with a single group operation on theciphertext space, since ri,1 = E(mi) · ri−1,1.

3. When an interval of receipt values [k..l] in a table needs to be aggregated, the server simplycomputes the sum by performing a = (rl,1 · rk,0 · rk,2); this intituitively can be seen as thesum from row 1 to row l, plus the value in row k, minus the sum from row 1 to row k, usingthe homomorphic property.

The correctness of this scheme for fast aggregation follows from the additive homomorphismof the algorithm. Recall that an additive homomorphism is defined by E(m1) ·E(m2) = E(m1+

m2), using this definition in our construction for rk,1 and rl,1 yields

rk,1 =k∏

j=1

rj,0 = E

⎛

⎝

k∑

j=1

mj

⎞

⎠ (5.11)

rl,1 =l∏

j=1

rj,0 = E

⎛

⎝

l∑

j=1

mj

⎞

⎠ (5.12)

also by using the fact that a homomorphism maps inverses in the ciphertext space to inverses inthe plaintext space:

E(mj)−1 = E(−mj) (5.13)

we obtain the following expression for rk,2

rk,2 = E

⎛

⎝

k∑

j=1

−mj

⎞

⎠ = E

⎛

⎝−k∑

j=1

mj

⎞

⎠ . (5.14)

Finally we apply

a = rl,1 · rk,0 · rk,2

= E

⎛

⎝

l∑

j=1

mj

⎞

⎠ · rk,0 ·E

⎛

⎝−k∑

j=1

mj

⎞

⎠

= E

⎛

⎝mk +l∑

j=k+1

mj

⎞

⎠

= E

⎛

⎝

l∑

j=k

mj

⎞

⎠ (5.15)

which proves the algorithm computes the encrypted aggregated sum of the range [k..l].


Instantiating the previous construction for Castelluccia’s algorithm consists of a modular addi-tion moduloM , which is in the order of key sizes secure for symmetric ciphers; finding the inverseonly requires one sum. In Paillier’s construction, an aggregation consists of a multiplication anda division in a group where factoring integers is hard; while finding a multiplicative inverse, isproportional to the logarithm of the modulo times the complexity of the arithmetic operations.

In terms of security, this construction only uses the output of the semantically secure functionE, to compute functions of polynomial complexity; therefore, an attacker A is not more successfulin performing an attack against a table with only outputs of E than it is performing an attack forthe fast aggregation table, since A by having any of the two versions could compute the other inpolynomial time.

This construction can be seen as an application of Run-length encoding from the field ofdata compression. The idea of run length encoding consists in taking advantage of consecutivecharacters with the same value to perform compression. Consider the following string in thebinary alphabet, and its transformation using Run-length encoding:

11111111111111110000000000111111111 → 1, 16, 10, 9 (5.16)

the transformed string represents the original chain in a more compact way, by saying it starts withone repeated 16 times, then 10 zeros and finally 9 ones.

If we see the string not as data, but as an array where a one on the i-th position represents thatthe i−th row must be added to the final result. Then, the proposal can compute the result in lessoperations, than if each part is added independently. In particular for this chain, using the modifieddatabase structure, a query of this form could be computed using just one homomorphic operation.

We will know prove our construction is asymptotically better than a construction based onbuckets. In the proof, we will derive an upper bound for the expected number of operations inboth algorithms, to show ours is lower. We do not claim the bound we derive for our proposal istight.

Theorem 6. In average, the proposed method makes less calls to the homomorphic aggregationfunction than bucket-based methods.

Proof. We will make use of the binary string representation for the rows returned by a query.

Let p > 0 be the probability of finding a 1 for a given bucket, in case p = 0 the result containsno numbers to aggregate, so the algorithm returns 0. For the remaining values of p, there are twocases to consider:

• If 0 < p ≤ 0.5, then it is faster to add the numbers one by one. Therefore, for this case theexpected number of additions to perform is given by:

L

B(p · 1 ·B) = Lp (5.17)

where L is the length of the table and B is the length of the bucket.

5.5. PROPOSAL 119

• if 0.5 < p ≤ 1, then the fastest way to proceed is to use the bucket and then subtract the 0sof the string one by one, like this:

L

B(1 + (1− p) · 1 ·B) =

L

B+ L(1− p) (5.18)

The results follow from the definition of expected value. L/B represents the number of bucketsthat must be accessed to compute the final result. Now we will compute the expected number ofoperations for our method.

When a query is to be aggregated, the method does the following:

1. Process bit by bit, and wait for a number of consecutive 1s, call this string s1 and |s1| itslength.

2. Once we know the length of s1, the number of operations is given by the following countingrule:

c(x) =

⎧

⎨

⎩

1, if |s1| = 1.

2, otherwise.

If the number of 0s exceeds the number of ones, one could negate the string, compute the resultand subtract it from the string containing 1s only. Hence, our function is symmetrical with respectto p = 0.5 in the asymptotic case, since the additional addition vanishes as L grows.

Given the previous counting procedure, the expected number of operations is given by:

E[c(x)] =L ·

(

p · (1− p) · 1 +L∑

i=2

(

pi · (1− p) ·2

i

))

(5.19)

=L ·

(

p · (1− p) + 2 · (1− p) ·L∑

i=2

(pi

i

))

(5.20)

=L · (1− p)

(

p+ 2 ·

(L∑

i=1

(pi

i

)

− p

))

(5.21)

The reasoning behind the equation is that we expect to have length one chains, that can be addedwith one addition with probability p(1−p); for all the other lengths of the chain, we can add themusing two operations. In addition, if |s1| = L then the number of operations is 1 since we cansimply return the last aggregate value in the table.

We will now show our method has better average behavior. We will need the Maclaurin seriesexpansion for the logarithm.

− log(1− x) =∞∑

i

xi

i(5.22)


which is valid for −1 ≤ x < 1. Since p = 1 is handled as a special case in our algorithm, we canuse this series in (5.21), which yields

E[c(x)] <L · (1− p) · (p+ 2 · (− log (1− p)− p)) (5.23)

<L · (1− p) · (−2 · log (1− p)− p) (5.24)

Which allows us to solve the following inequality:

E[c(x)] < L · (1− p) · (−2 · log (1− p)− p) < L · p (5.25)

which holds for 0 < p < 1. Therefore, our construction in average computes less operations thanmethods based on buckets for all 0 < p < 1. For the extreme points p = 0 or p = 1, the resultsmight be equal depending on the length of the bucket. Since our construction is symmetric andthe bucket-based one has worse behavior for p > 0.5, the inequality also applies.

Using (5.25), we can derive the values for the methods in the worst case as 0.443147L for oursand 0.5L for the bucket-based one. Experimental results shown in Fig. 5.10 in Section 5.7, presentsimulations supporting both the fact that our upper bound is not tight, and that our method has abetter average behavior. In those simulations, the worst case result for our method was 0.375L,while the result for the bucket construction was 0.487L.

A 2−dimensional extension to the algorithm is also possible, we present a way to implementit on a database. The idea is to extend the concept of lines we could compute efficiently in the1−dimensional scenario, to rectangles in two dimensions. First, the binary string is modelled as amatrixM , with a preselected number of columns.

The memoization strategy consists in letting Mi,j represent the sum from the rectangle withcorners at M1,1 and Mi,j in the original data. To compute a given rectangle of width w andheight h ending at coordinates (i, j), one can proceed as follows: add Mi,j and then subtractthe rectangles above (Mi−h,j) and to the left of it (Mi,j−w); since subtracting these rectanglessubtracts Mi−h,j−w twice, we can add it to get the right result. The procedure is illustrated in thefollowing example using matrices:

⎡

⎢⎢⎢⎢⎢⎢⎣

0 0 0 0 0

0 0 0 0 0

0 0 1 1 1

0 0 1 1 1

0 0 0 0 0

⎤

⎥⎥⎥⎥⎥⎥⎦

=

⎡

⎢⎢⎢⎢⎢⎢⎣

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

0 0 0 0 0

⎤

⎥⎥⎥⎥⎥⎥⎦

+

⎡

⎢⎢⎢⎢⎢⎢⎣

−1 −1 −1 −1 −1−1 −1 −1 −1 −10 0 0 0 0

0 0 0 0 0

0 0 0 0 0

⎤

⎥⎥⎥⎥⎥⎥⎦

+

⎡

⎢⎢⎢⎢⎢⎢⎣

−1 −1 0 0 0

−1 −1 0 0 0

−1 −1 0 0 0

−1 −1 0 0 0

0 0 0 0 0

⎤

⎥⎥⎥⎥⎥⎥⎦

+

⎡

⎢⎢⎢⎢⎢⎢⎣

1 1 0 0 0

1 1 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

⎤

⎥⎥⎥⎥⎥⎥⎦

(5.26)

5.5. PROPOSAL 121

bj S1 S2 S3

{f(1)}

{f(2)}

{f(3)}

(a) Store

bj S1 S2 S3

{Q}{a1}

{a2}

{a3}

(b) Query ({Q} is equal for all storage servers)

Figure 5.7: Diagram of the scenario using Shamir’s Secret Sharing Scheme, for three independentstorage servers.

As a final remark about the aggregation techniques, it is important to mention the techniqueonly guarantees data privacy; authenticity is outside its scope. In addition, an attacker runningsoftware on the outsourced server, might be able to infer the number of aggregated ciphertexts, bychecking the cpu usage of the machine.

5.5.4 Scenario Using Shamir’s Scheme

Unlike the previous scenarios, where security relies on a cryptographic key; the construction basedon Shamir’s cryptosystem does not require a key for aggregating information.

When a transaction is going to take place, bj creates a random polynomial f(x), whose freeterm is the actual price to be payed; the degree of the polynomial is one less than the number ofindependent storage servers available s. bj evaluates the polynomial at the points S1, . . . , Ss whereeach Si is the identification number of a given server; this number will always be the same eachtime information is stored. After evaluating the polynomial, each server stores its correspondingimage f(Si). The evaluation of the polynomial is performed in Zp, where p is a prime numberlarge enough, such that the sum does not overflow.

When information needs to be queried ui sends the same query Q to all the storage servers;each server replies with the aggregate sum of its corresponding items ai. This sum is performedon the integers (Z); this is very convenient, since the logic from the storage does not need to bemodified. When ui obtains all the ai values, it divides each value by p taking the residue; then itperforms polynomial interpolation as shown in 5.4.3 to recover the result of the sum.

Correctness of the scheme follows from the polynomial properties exposed in section 5.4.3.The only detail not addressed by the properties, is the fact that a sum at Si might overflow theunderlying data type used by the storage system. In that case the result will be erronous, sinceinformation was lost when the overflow happened. For the case of an expense tracking application,if a 64 bit data type is used to store the numeric data, and we use a prime p less than but close to232; the maximum number of retrieved items that could be safely added would be 232 transactionsor items, depending on the granularity of the stored information. In general if the data type isof size 2w and p < 2v where v < w, then we can add 2w−v different quantities correctly. Asa final remark about the execution of the scheme, it is important to mention that, the security ofthe system does not rely on the size of p, since the secret is distributed uniformly in the range[0 . . . p− 1]; however compromise of all the servers would render the scheme insecure.


5.6 Experimental Results

In this section, we present the results of the implementation of the achemes presented in Section5.4; all the algorithms were tested using an Intel Core 2 Quad Q6700 (8M Cache, 2.66 GHz, 1066MHz FSB) in a 64 bit Windows environment, using the Java JDK 1.6 release 21. The choice ofthe Java environment for our experiments obeyed to its widespread used in database applicationsworldwide, since the purpose of our experiments is to recreate how an independent business couldadd itself to the service, by implementing additional routines to its existing accounting software.

To implement the symmetric algorithm, we selected the AES cipher in electronic codebookmode (with r treated as a counter) as the pseudo random function, the key size used was 128 bits,this size was also selected for the modulus M ; since the output does not need to be reduced, thelength preserving hash function was set to be h(x) = x. We argue that using AES as a pseudorandom function (PRF) is secure for our scenario because of the following reasons:

1. Assuming we have P which is a secure PRP (pseudo random permutation) in {0, 1}n, thenP is a (q, q2/2n+1)-secure PRF [116], this is: given any algorithm A that queries P anumber q of times, it cannot distinguish the output of P from a true random permutation F ,with an advantage greater than q2/2n+1, where the advantage function is defined as:

|pr[A(P ) = 1]− pr[A(F ) = 1]| (5.27)

as it is proved in [117] (theorems 1 and 2), this notion of security has equivalent strength tothe notion of security presented in Section 5.2 for our problem. This justifies the use of ablock cipher (AES) as a PRF; even further, this bound can be used to determine how manytimes a block cipher can be treated as a PRF, depending on its cryptographic strength.

2. In our scenario the number of queries to the block cipher q is negligible, compared to the 2n

output space of the block cipher.

3. In the protocol, the ECB mode under a same key is never used to encrypt the same valuetwice.

Therefore, if our construction using AES is insecure, it means that an adversary has an advantagethat is non negligible for a small number of queries2, contradicting the assumption that AES is asecure PRP for a small q. Even though more secure constructions exist, we wanted to keep theimplementation as simple as possible to reduce costs in a real scenario.

To assess the security of the previous argument in practice, we tested the output of our im-plementation with the statistical tests for cryptographic number generators from NIST [118]. Thetests were performed using 20000 bits of data for each test, each test was run 1000 times withdifferent outputs. The outcome of a single run is a P − value, if this value is greater than 0.01,that particular run is considered as passed, and failed otherwise. The number of passed runs foreach test type is summarized in Table 5.1.

According to the description of the software, for 1000 runs of a particular type of test, theminimum requirement for randomness is 980 passed runs. To derive this proportion, the authors

2The expected number of queries is the number of times one user uses the same key.

5.6. EXPERIMENTAL RESULTS 123

Table 5.1: Results of randomness of our implementation using the NIST suite of tests 1000 times.

Test Passed TestsFrequency 994

Block Frequency 994Cumulative Sums 993

Non-Overlapping Template 982Runs 994

Longest Run 987Rank 991FFT 984

Overlapping Template 1000Universal 1000

Approximate Entropy 1000Serial 996

Linear Complexity 985

of the software considered α = 0.01 as significance level, to compute the confidence interval:

p+− 3

√

p(1− p)

m(5.28)

where p = 1 − α and m is the number of tests performed. The outcome of the simulation showsthat the construction meets the required proportion of passed runs for all the different types of testsapplied, confirming experimentally that this construction could be used in practice; despite this,the result should be interpreted as “this set of tests, found no evidence of non-random behavior”rather than “the tests show the output is random”, because the latter would rely on unproven claimsrelated to AES security.

For the implementation of Paillier’s cryptosystem the algorithm was implemented using theBigInteger class available at the JDK 1.6. Regarding the randomness of the implementation, as itwas noted in Section 5.4.2 it is based on the hardness of the underlying problem, that is why thestatistical tests were not necessary.

To test the algorithms, we created a sample application for a set of fictional restaurants whocan input to our database. The application takes an order consisting of a meal and then uploadsthe results to a database. For simplicity the application contains one table, whose structure issummarized in Table 5.2. The table contains several fields and shows the interaction among thedifferent algorithms presented. The first rows contains the order of data created by a particularuser; since we aggregate data incrementally, we need to know rows are continuous in the databaseto compute the correct result. The second column contains dates ordered by date, using order pre-serving encryption; we assumed for our application that the user could provide the encryption ofthe date using his secret key. The third column has the name of the restaurant, which is encryptedusing and efficient searchable encryption, which allows efficient exact queries. Finally the actualamount paid is encrypted using any of the schemes from Section 5.4, the column.

A table like this allows to query the the result of money spent at a particular place and alsoamount of money paid in a given a period of time. A proxy using CryptDB could create such


Table 5.2: Table structure for the sample application.

User order Date Business User Total1 OPE(OuA, 2012/12/23) ESE(PuA, R.A) uA E(P ′uA, 25)1 OPE(OuB , 2012/12/27) ESE(PuB , R.B) uB E(P ′uB , 20)2 OPE(OuA, 2012/12/28) ESE(PuC , R.C) uA E(P ′uA, 15)1 OPE(OuC , 2012/12/31) ESE(PuA, R.A) uC E(P ′uC , 7)3 OPE(OuA, 2012/12/29) ESE(PuC , R.C) uA E(P ′uA, 30)

0

20

40

60

80

100

120

140

160

180

0 10000 20000 30000 40000 50000

Tim

e (m

s)

Aggregate ciphertexts

Paillier (1024)Castelluccia et al. (128)

Shamir (64)

Figure 5.8: Decryption times in our implementation of the three algorithms.

a table like this. Note that the values are only sorted according to date for a single user, sinceencryption with different keys for the order preserving functions are unrelated.

We first performed experiments focusing on the decryption operation; results of this experi-ment are summarized in Fig. 5.8. The fact that the running time of the symmetric algorithm islinear in the number of elements involved in the aggregation operation, does not allow it to scalewell when the number of values involved in an aggregation is large.

From the results we can infer that when user interaction is not a problem, the symmetricapproach could help with implementation costs as long as the number of aggregations that mustbe performed from a single query is not too large; however if the number of decryptions that mustbe performed is large, the single step Paillier decryption enhances decryption times significantly,but requires better terminals at POS; both encryption and decryption times averaged 19 ms fora 1024 bit key. If the possibility of several non-colluding servers is available (e.g. via severalcloud providers), Shamir’s construction outperforms the encryption algorithms by an order ofmagnitude; our sample polynomial interpolation algorithms using 64 bit integers in java had 20.7µs as average execution time. This was an expected result given that the complexity for thisscheme depends on the number of servers and not the number of aggregate ciphertexts.

Even though the communication factor was not taken into consideration, another drawback ofthe symmetric approach is that it needs to know how many decryptions must be made with eachkey in order to output a correct result.

We also performed tests evaluating the computational load at the outsourced untrusted server,the simulation results can be seen in Fig. 5.9. As it was expected Paillier’s cryptosystem hadthe slowest throughput by a long margin; this was expected because of the size of the numbers

5.6. EXPERIMENTAL RESULTS 125

0

500

1000

1500

2000

2500

0 5000 10000 15000 20000 25000 30000 35000 40000

Tim

e (m

s)

Aggregate ciphertexts

Paillier (1024 bits)Castelluccia et al. (128 bits)

Shamir (64 bits)

Figure 5.9: Aggregation times in our implementation of the three algorithms.

Table 5.3: Comparison of the aggregation schemes.

Scheme State Security Speed Db compatible OverheadPaillier [23] No Key Slow No Large

Castell. et al. [25] Yes Key Medium No LowShamir [44] No Replication Fast Yes Low

involved for the homomorphic operation (2048 bits). Addition in Castelluccia et al. was fasterthan Paillier’s due to two factors, the first of them is the homomorphic operation, which is modularaddition; the second is the size, which was considerably smaller than Paillier’s. Finally, Shamir’sconstruction reduces to normal addition which is considerably faster since basic data types can beused.

To provide a guide about which algorithm, suits best a particular situation, we summarize theresults of the experiment on Table 5.3. The meaning of each column is as follows: state, mentionsif the algorithm needs information about the rows returned, to perform decryption; security tells ifthe algorithm is based on a key or the replication mechanism to achieve security; speed is a sub-jective classification based on the experiments about aggregation times in the sample application.Database-compatible tells if the system can be used with standard SQL functions, over standardnumber data types; it is a way to summarize if data can be summed using the SUM function;finally overhead, tells how much overhead is introduced by the data types used by each algorithm.

The previous results about the aggregation times for Paillier’s cryptosystem, show that ourconstruction from Section 5.5.3 is relevant in practice. Continuing with the example application;when the values from column “User order” from Table 5.2 do not have any gaps, the results can bereturned with two modular multiplications, regardless of the number of ciphertexts to aggregate.This can be seen as a bucket with no range, which is significantly more practical than the con-struction from [45] for queries involving extensive sums. Table 5.4 shows the relevant columns, toimplement the aggregation algorithm on Table 5.2; the last column could be replaced by a singlecomputation, by multiplying the column containing the precomputed values “Precomp. Agg” byan encrypted version of −1 to save space.

The result of using the proposal can be seen in Fig. 5.10, where the naive approach of multi-plying each number independently, is compared with the proposal using one and two dimensions.


Table 5.4: Table implementing fast aggregation for the sample application.

User order User Total Precomp. Agg Inverse1 uA E(P ′uA, 25) E(P ′uA, 25) E(P ′uA,−25)1 uB E(P ′uB , 20) E(P ′uB, 20) E(P ′uB,−20)2 uA E(P ′uA, 15) E(P ′uA, 40) E(P ′uA,−40)1 uC E(P ′uC , 7) E(P ′uC , 7) E(P ′uC ,−7)3 uA E(P ′uA, 30) E(P ′uA, 70) E(P ′uA,−70)

0

20000

40000

60000

80000

100000

0 0.2 0.4 0.6 0.8 1

Nu

mb

er o

f h

om

om

orp

hic

op

erat

ion

s

Probability of retrieving a row

Encryption Times

Naive approachProposal 1DProposal 2D

1000 Row bucket

Figure 5.10: Number of homomorphic operations performed when instantiating the proposal withrespect to the percentage of total rows retrieved for a user.

The test represented in the figure, considered a user with 100000 records in the database; for the2−D proposal, data was modeled as a 100 row by 1000 column matrix. The implementation forthe 2−D scenario used a greedy strategy, where for a given point in the matrix, the greatest rectan-gle starting at that point was computed. This was only applied if the rectangle had an area greaterthan 3, because using the 2-D algorithm on smaller rectangles would be less efficient, than addingthe numbers individually.

As it was expected from the definition of the proposal, the number of multiplications increasesas the entropy in the positions of the retrieved record does; retrieved records are represented using1s in a binary string, whose length is the number of records for a particular user. Following thetheoretical model, the maximum number of operations arises when the entropy function is at itsmaximum, which in the figure happens when the probability of a record being retrieved equals0.5. For this scenario, the 1-dimensional proposed method, made 23.3 % less invocations to thehomomorphic function, than the bucket-based construction from [45].

5.7 Conclusions

We presented in this chapter three architectures for a secure outsourced database, that performsqueries where financial data is aggregated using homomorphic encryption. For each key paradigm,we showed how different homomorphic cryptosystems and linear secret sharing schemes, can besuitable for different parts of the system, in terms of user convenience, computational load andinfrastructure acquisition.

5.7. CONCLUSIONS 127

We also presented a dynamic programming algorithm of theoretical interest, that takes ad-vantage of the properties of homomorphic public key cryptosystems, to accelerate the process ofaggregating data; it achieves this by using additional storage or memory at the outsourced server.We proved the algorithm is correct, secure and has a better asymptotic behavior than the construc-tion based on buckets from [45]. The construction is interesting since it shows, that the problem ofencrypted databases needs to be treated in a holistic way. Even though efforts are being done fromthe security community; designs focusing on several parts of the application, namely database andcryptography, can achieve better results when working together.

Chapter 6

Conclusions and Future Work

129

130 CHAPTER 6. CONCLUSIONS AND FUTUREWORK

The objective of this research has been to provide better performance and integrity mecha-nisms for encoded information. Results of this work can be used for several real world scenariosincluding: information transmission using Network Coding, storage networks using RegeneratingCodes and outsourced databases. This chapter lists the major contributions of this dissertation.

6.1 Contributions

In Chapter 2 we study and propose solutions to some security issues related to pollution attacks inNetwork Coding in the form of two schemes:

1. A practical scheme to identify nodes performing pollution attacks in a network. The mainadvantage of the scheme compared to the existing ones, is that it does not need a trustedauthority to perform blacklisting of misbehaving nodes. This scheme is presented in Section2.5.

2. A pollution detection scheme for XOR Network Coding without homomorphic functionswhich is suitable for small generations. The main advantage of this scheme is that its secu-rity is not limited by the small size of F2 which makes existing constructions very inefficientin this scenario. This scheme is presented in Section 2.6.

In Chapter 3 the focus was made on Regenerating Codes and Proofs of Data Possession. Themain goal is in this chapter is to design mechanisms to verify that an encoded block of a file isa valid codeword from an erasure code. This is a problem similar to pollution detection with theadditional constraint that the verifier does not have access to the codeword that is verifying due toits length. The following contributions were made in this chapter:

1. Presented two schemes to verify that an encoded file is available at an untrusted server.One of the constructions can be used an unbounded number of times (Section 3.3.1). Thesecond one even though faster, can only be used a bounded number of times (Section 3.3.2).Their advantages compared to existing schemes are: first, they can be used on files encodedby an erasure code; second, they address security attacks during the regeneration stage notconsidered by other proposals.

2. For an scenario where servers store parts of a file and where the number of servers is dy-namic, Section 3.3.5 presents an architecture that guarantees files are recoverable when atleast k servers are available. Advantages of this architecture include minimization of band-width to repair failed nodes and/or create new ones to satisfy user demand. In addition,verification can be done in polynomial time unlike existing schemes. The proposed archi-tecture can be used in Content Distribution Networks (CDN).

3. Presented and proved facts about an algorithm to enhance the effectiveness of Proofs ofData Possession in Section 3.3.7. The algorithm, reduces the amount of information thatneeds to be transmitted to find all the defective blocks at a remote server. The algorithm isthe first of its kind for verifying outsourced storage to the best of our knowledge. Regardingthe amount of transmitted information, it outperforms generic methods for finding defectiveelements, by taking advantage of the linear properties of some existing schemes. We show

6.2. FUTUREWORK 131

the effectiveness of the approach used by the algorithm via simulation, in a problem relatedto remote synchronization of music content in Section 3.3.6.

In Chapter 4 we worked on preventing a type of attack known as a Diversity Attack. Inaddition, we proposed a new Attack for Network Coding that we named Decodability Attackand a scheme to prevent it. The attack occurs in networks where nodes are required to decodeimmediately after a packet has been received. In this chapter we make the following contributions:

1. Proposed a scheme in Section 4.5 to detect diversity attacks. The scheme is less compu-tationally intensive and with less transmission overhead than the existing approaches notinvolving randomization.

2. Proposed a new scenario for diversity attacks in storage networks and presented a solutionto the problem in Section 4.6.

3. Introduced Decodability Attacks and proposed a solution in Section 4.7. The solution is tothe best of our knowledge the first one to this problem.

In Chapter 5, the goal was to improve existing approaches to perform computations on encodeddata at an outsourced server. The context for the problem was set to that of expense trackingsoftware. The contributions of this chapter are the following:

1. Presented 3 architectures to add information securely at an outsourced server. In particu-lar, the one based on public key homomorphic encryption from Section 5.5.2, outperformsexisting methods based on bucketization.

6.2 Future Work

During this work, great effort was made to make humble contributions to transmission, storage andcomputations with encoded data. In this section, we will describe some some interesting directionsfor future work aimed at improving some of the shortcomings of our proposed constructions.

In Chapter 2 we proposed a system for pollution detection for XOR Network Coding basedon Cryptographic Accumulators. The system is able to simulate the linearity of homomorphicsignatures by performing additional computations at the source generating the packets. To makethe computation trade-off for the source reasonable, we designed a strategy that achieves logarith-mic retransmission overhead while still guaranteeing verifiability to the receivers. However, thisstrategy does not work well for generations of large size. Compared to other proposals in the area,our construction is very efficient at the verifiers and it has constant overhead as packets travel thenetwork. For this reason, designing pollution prevention schemes for XOR network coding thatcan work for any generation size, constitutes an interesting research problem in itself. Regard-ing our identification technique for nodes introducing pollution, designing decentralized schemeswith lower overhead is a very interesting research problem. In particular for our construction, theconstraint comes from providing efficient multicast authentication.

In Chapter 3 we presented several primitives related to possession of outsourced encodedinformation. One of them is based on a scheme by Shacham and Waters [15]. Its advantage isthat the secret function can be used many times, but it has a trade-off between the amount of

132 CHAPTER 6. CONCLUSIONS AND FUTUREWORK

information transmitted for each verification. On the other hand, our one-time-use protocol hasvery compact verification and storage requirements. Creating a scheme that has the advantages ofboth constructions presents interesting challenges for practical deployments of these primitives.

To increase the amount of information offered by the schemes, we proposed an algorithm thatnot only finds that some blocks are missing, but it can actually find exactly which blocks. Thisproblem falls in the realm of a discipline called Group Testing where the goal is to find a set ofdefective elements in a population. Our construction is able to outperform existing generic grouptesting constructions. The reason for this, is that generic algorithms do not assume any relationshipamong the results of different tests. In our case, we can take advantage of the linear properties ofexisting protocols for data possession. For the worst case in our algorithm, we must send a numberof test results equal to the number of blocks that were tested. This raises the question of whetherit is possible to find schemes that can provide an estimate about all the blocks with sublineartransmission overhead for the worst case.

In Chapter 4 a solution to the diversity problem in network transmission using network codingwas presented. Even though the solution does not need to transmit the payload of the packetsfor verification, it still sends the network coding coefficients from all the parents to the children.Therefore, developing techniques that allow a node to detect diversity attacks immediately withoutreceiving the coding coefficients would reduce significantly the overhead of this kind of system.The same challenge applies to protocols aimed at detecting decodability attacks. An additionalchallenge is to integrate the solution with network-wide network monitoring schemes.

In Chapter 5 we proposed a system to improve computations in outsourced data. The fastest ofthem involved applying an erasure code and sending the data to several servers. This approach hasthe same performance as a database without any kind of security. Unfortunately, it significantlyincreases the storage costs for applications. Finding schemes that can provide similar levels ofperformance for a single server is a challenging problem for this area. For the single server sce-nario, the public key construction based on memoization can be used for the single server scenariosecurely. However, its performance for databases where records are deleted or updated decreases.A useful extension for this construction could include data structures to handle this class of oper-ations efficiently.

Appendix A

List of Author’s Publications andAwards

A.1 Journals

1. J. Corena and T. Ohtsuki, “Secure and Fast Aggregation of Financial Data in Cloud-BasedExpense Tracking Applications,” Journal of Networks and Systems Management, vol. 20,no. 4, pp. 534-560, July 2012.

2. J. Corena and T. Ohtsuki, “Pollution-Free Regenerating Codes With Fast ReconstructionVerification for Verifiable Cloud Storage,” Journal of Networks. (accepted for publication)

A.2 Full Articles on International Conferences Proceedings

1. J. Corena and T. Ohtsuki, “AMultiple-MAC-Based Protocol to Identify Misbehaving Nodesin Network Coding,” In Proceedings of the 2012 IEEE 76th Vehicular Technology Confer-ence (VTC2012-Fall), pp. 1-5, Quebec City, Canada, Sept. 2012.

2. J. Corena and T. Ohtsuki, “Thwarting Diversity Attacks in Wireless Network Coding UsingThreshold Signatures and a Sender-Centered Approach,” In Proceedings of the 2012 IEEEGlobal Communications Conference (GLOBECOM), pp. 1060-1065, Anaheim, USA, Dec.2012.

3. J. Corena and T. Ohtsuki, “Eavesdropper Resilient Network Coding Using Random Substi-tutions of the Global EncodingMatrix,” In Proceedings of The 10th IEEE Vehicular Technol-ogy Society Asia Pacific Wireless Communications Symposium (IEEE VTS APWCS 2013),pp. 46-50, Aug. 2013.

4. J. Corena and T. Ohtsuki, “Load Balancing Regenerating Codes for Multimedia ContentStreaming,” In Proceedings of the 2013 IEEE 24th International Symposium on PersonalIndoor and Mobile Radio Communications (PIMRC), pp. 3558-3562, London, UK, Sept.2013.

133

134 APPENDIX A. LIST OF AUTHOR’S PUBLICATIONS AND AWARDS

5. J. Corena and T. Ohtsuki, “Proofs of Data Possession and Pollution Checking for Regenerat-ing Codes,” In Proceedings of the 2013 IEEEGlobal Communications Conference (GLOBE-COM), pp. 2739-2744, Atlanta, USA, Dec. 2013.

6. J. Corena, A. Basu, S. Kiyomoto, Y. Miyake and T. Ohtsuki, “XOR Network Coding Pollu-tion Prevention Without Homomorphic Functions,” In Proceedings of the 11th Annual IEEEConsumer Communications and Networking Conference (CCNC), pp. 544-551, Las Vegas,USA, Jan. 2014.

7. A. Basu, J. Corena, S. Kiyomoto, S. Marsh, J. Vaidya, G. Guo, J. Zhang and Y. Miyake,“Privacy Preserving Trusted Social Feedback,” In Proceedings of the 29th ACM Symposiumon Applied Computing 2014 (ACM SAC), pp. 1706-1711, Gyeongju, Korea, Mar. 2014.

8. A. Basu, A. Monreale, J. Corena, F. Giannotti, D. Pedreschi, S. Kiyomoto, Y. Miyake, T.Yanagihara and R. Trasarti, “A Privacy Risk Model for Trajectory Data,” 8th IFIP WG 11.11International Conference on Trust Management 2014 (IFIPTM14), Singapore, July 2014.(to appear)

A.3 Short Articles on International Conferences Proceedings

1. A. Basu, J. Corena, S. Kiyomoto, J. Vaidya, S. Marsh and Y. Miyake, “PrefRank: FairAggregation of Subjective User Preferences,” In Proceedings of the 29th ACM Symposiumon Applied Computing 2014 (ACM SAC), pp. 287-288, Gyeongju, Korea, Mar. 2014.

A.4 Articles on Domestic Conference Proceedings

1. J. Corena and T. Ohtsuki, “Pollution Attacks in XOR Network Coding: a Broadcast En-cryption Perspective,” Proceedings of the IEICE General Conference 2011, A-6-3. Mar.2011.

2. J. Corena and T. Ohtsuki, “A Decentralized Non-Repudiation Protocol to Identify Misbe-having Nodes in Network Coding,” Technical Committee on Radio Communication Systems(RCS), IEICE Tech. Rep., vol. 111, no. 404, RCS2011-313, pp. 263-268, Jan. 2012.

3. J. Corena and T. Ohtsuki, “A Cooperative MAC-Based Protocol for Blacklisting in NetworkCoded Networks,” Technical Committee on Radio Communication Systems (RCS), IEICETech. Rep., vol. 112, no. 132, RCS2012-84, pp. 55-60, July 2012.

4. J. Corena and T. Ohtsuki, “Security Challenges in Network Coding: The case of the Diver-sity Problem,” Technical Committee on Radio Communication Systems (RCS), IEICE Tech.Rep., vol. 112, no. 443, RCS2012-365, pp. 483-488, Feb. 2013.

5. J. Corena, A. Basu, S. Kiyomoto, Y. Miyake and T. Ohtsuki, “Decodability Attacks in XORNetwork Coding,” Proceedings of the 2013 IEICE Society Conference, BS-7-44, Sept. 2013.

A.5. AWARDS 135

A.5 Awards

1. IEEEVTS japan 2012 Young Researchers Encouragement Award (Awarded paper: J. Corenaand T. Ohtsuki, “A Multiple-MAC-Based Protocol to Identify Misbehaving Nodes in Net-work Coding,” In Proceedings of the 2012 IEEE 76th Vehicular Technology Conference(VTC2012-Fall), pp. 1-5, Quebec City, Canada, Sept. 2012.)

2. IEEE APWCS 2013 Best Student Paper Award (Awarded paper: J. Corena and T. Ohtsuki,“Eavesdropper Resilient Network Coding Using Random Substitutions of the Global En-coding Matrix,” In Proceedings of The 10th IEEE Vehicular Technology Society Asia PacificWireless Communications Symposium (IEEE VTS APWCS 2013), pp. 46-50, Aug. 2013).

136 APPENDIX A. LIST OF AUTHOR’S PUBLICATIONS AND AWARDS

References

[1] S.-Y. Li, R. W. Yeung, and N. Cai, “Linear network coding,” Information Theory, IEEETransactions on, IEEE, vol. 49, no. 2, pp. 371–381, 2003.

[2] C. Gkantsidis and P. Rodriguez, “Network coding for large scale content distribution,” inINFOCOM 2005, Proceedings, IEEE, vol. 4, pp. 2235–2245, 2005.

[3] S. Katti, H. Rahul, W. Hu, D. Katabi, M. Medard, and J. Crowcroft, “Xors in the air:Practical wireless network coding,” Networking, IEEE/ACM Transactions on, IEEE/ACM,vol. 16, no. 3, pp. 497–510, Jun. 2008.

[4] C. Fragouli and A. Markopoulou, “A network coding approach to overlay network moni-toring,” in 43rd Annual Allerton Conference on Communication, Control, and Computing,pp. 28–30, 2005.

[5] M. N. Krohn, M. J. Freedman, and D. Mazieres, “On-the-fly verification of rateless erasurecodes for efficient content distribution,” in Security and Privacy, 2004. Proceedings. 2004IEEE Symposium on. IEEE, pp. 226–240, 2004.

[6] C. Gkantsidis and P. Rodriguez, “Cooperative security for network coding file distribution,”in INFOCOM 2006, Proceedings, IEEE, vol. 3, pp. 5, 2006.

[7] R. A. Popa, A. Chiesa, T. Badirkhanli, and M. Medard, “Going beyond pollution attacks:Forcing byzantine clients to code correctly,” CoRR, vol. abs/1108.2080, 2011.

[8] J. Corena, A. Basu, S. Kiyomoto, and T. Miyake, Y. Ohtsuki, “Decodability attacks in xornetwork coding,” in Proceedings of the 2013 IEICE Society Conference, Sep. 2013.

[9] A. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran, “Network coding fordistributed storage systems,” Information Theory, IEEE Transactions on, vol. 56, no. 9, pp.4539–4551, Sep. 2010.

[10] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” Journal of theSociety for Industrial and Applied Mathematics, vol. 8, no. 2, pp. 300–304, 1960.

[11] K. Rashmi, N. B. Shah, D. Gu, H. Kuang, D. Borthakur, and K. Ramchandran, “A solutionto the network challenges of data recovery in erasure-coded distributed storage systems: Astudy on the facebook warehouse cluster,” in Proceedings of 5th USENIX Workshop on HotTopics in Storage and File Systems. USENIX, pp. 8–8, 2013.

137

138 REFERENCES

[12] M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, “Xor-ing elephants: Novel erasure codes for big data,” Proceedings of the VLDB Endowment,vol. 6, no. 5, 2013.

[13] B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Sri-vastav, J. Wu, H. Simitci et al., “Windows azure storage: a highly available cloud storageservice with strong consistency,” in Proceedings of the Twenty-Third ACM Symposium onOperating Systems Principles. ACM, pp. 143–157, 2011.

[14] G. Ateniese, R. Burns, R. Curtmola, J. Herring, L. Kissner, Z. Peterson, and D. Song,“Provable data possession at untrusted stores,” in Proceedings of the 14th ACM conferenceon Computer and communications security, ser. CCS ’07. ACM, pp. 598–609, 2007.

[15] H. Shacham and B. Waters, “Compact proofs of retrievability,” in Advances in Cryptology- ASIACRYPT 2008, ser. Lecture Notes in Computer Science, J. Pieprzyk, Ed. SpringerBerlin / Heidelberg, vol. 5350, pp. 90–107, 2008.

[16] K. D. Bowers, A. Juels, and A. Oprea, “Hail: a high-availability and integrity layer for cloudstorage,” in Proceedings of the 16th ACM conference on Computer and communicationssecurity, ser. CCS ’09. ACM, pp. 187–198, 2009.

[17] Y. Dodis, S. Vadhan, and D. Wichs, “Proofs of retrievability via hardness amplification,” inTheory of Cryptography. Springer, pp. 109–127, 2009.

[18] C. Wang, Q. Wang, K. Ren, and W. Lou, “Privacy-preserving public auditing for data stor-age security in cloud computing,” in INFOCOM 2010, Proceedings. IEEE, pp. 1–9, 2010.

[19] Y. Zhu, G.-J. Ahn, H. Hu, S. S. Yau, H. G. An, and C.-J. Hu, “Dynamic audit services foroutsourced storages in clouds,” Services Computing, IEEE Transactions on, vol. 6, no. 2,pp. 227–238, 2013.

[20] R. Curtmola, O. Khan, R. Burns, and G. Ateniese, “Mr-pdp: Multiple-replica provable datapossession,” in Distributed Computing Systems, 2008. ICDCS’08. The 28th InternationalConference on. IEEE, pp. 411–420, 2008.

[21] C. Hanser and D. Slamanig, “Efficient simultaneous privately and publicly verifiable ro-bust provable data possession from elliptic curves,” in SECRYPT 2013, 10th InternationalConference on Security and Cryptography. SciTePress, pp. 15–26, 2013.

[22] A. Le and A. Markopoulou, “Auditing for network coding storage,” in International Sym-posium on Network Coding (NetCod), 2012.

[23] P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” inProceedings of the 17th international conference on Theory and application of crypto-graphic techniques, ser. EUROCRYPT’99. Berlin, Heidelberg: Springer-Verlag, pp. 223–238, 1999.

REFERENCES 139

[24] B. Thompson, S. Haber, W. G. Horne, T. Sander, and D. Yao, “Privacy-preserving compu-tation and verification of aggregate queries on outsourced databases,” in Proceedings of the9th International Symposium on Privacy Enhancing Technologies, ser. PETS ’09. Berlin,Heidelberg: Springer-Verlag, pp. 185–201, 2009.

[25] C. Castelluccia, A. C.-F. Chan, E. Mykletun, and G. Tsudik, “Efficient and provably secureaggregation of encrypted data in wireless sensor networks,” ACM Trans. Sen. Netw., vol. 5,pp. 20:1–20:36, Jun. 2009.

[26] J. Byers, M. Luby, and M. Mitzenmacher, “A digital fountain approach to asynchronousreliable multicast,” Selected Areas in Communications, IEEE Journal on, vol. 20, no. 8, pp.1528 – 1540, Oct. 2002.

[27] P. Maymounkov, “Online codes,” Technical Report TR2002-833, NYU., 2002.

[28] A. Le, A. S. Tehrani, A. G. Dimakis, and A. Markopoulou, “Instantly decodable networkcodes for real-time applications,” arXiv preprint arXiv:1303.7197, 2013.

[29] P. Zhang, Y. Jiang, C. Lin, H. Yao, A. Wasef, and X. Shen, “Padding for orthogonality:Efficient subspace authentication for network coding,” in INFOCOM, 2011 Proceedings,IEEE pp. 1026 –1034, Apr. 2011.

[30] D. Boneh, D. Freeman, J. Katz, and B. Waters, “Signing a linear subspace: Signatureschemes for network coding,” in Proceedings of the 12th International Conference on Prac-tice and Theory in Public Key Cryptography: PKC ’09. Springer-Verlag, pp. 68–87, 2009.

[31] S. Agrawal and D. Boneh, “Homomorphic macs: Mac-based integrity for network coding,”in Applied Cryptography and Network Security, ser. Lecture Notes in Computer Science.Springer Berlin / Heidelberg, vol. 5536, pp. 292–305, 2009.

[32] A. Le and A. Markopoulou, “Cooperative defense against pollution attacks in network cod-ing using spacemac,” CoRR, vol. abs/1102.3504, 2011.

[33] M. N. Krohn, M. J. Freedman, and D. Mazieres, “On-the-fly verification of rateless erasurecodes for efficient content distribution,” in Security and Privacy, 2004. Proceedings. 2004IEEE Symposium on. IEEE, pp. 226–240, 2004.

[34] R. Gennaro, J. Katz, H. Krawczyk, and T. Rabin, “Secure network coding over the integers,”in Public Key Cryptography–PKC 2010. Springer, pp. 142–160, 2010.

[35] Z. Yu, Y. Wei, B. Ramkumar, and Y. Guan, “An efficient scheme for securing xor networkcoding against pollution attacks,” in INFOCOM 2009, Proceedings, IEEE, pp. 406 –414,2009.

[36] Q. Wang, L. Vu, K. Nahrstedt, and H. Khurana, “Mis: Malicious nodes identificationscheme in network-coding-based peer-to-peer streaming,” in INFOCOM, 2010 Proceed-ings, IEEE, pp. 1–5, 2010.

140 REFERENCES

[37] R. C. Merkle, “Protocols for public key cryptosystems,” Security and Privacy, IEEE Sym-posium on, vol. 0, p. 122, 1980.

[38] A. Le and A. Markopoulou, “Nc-audit: Auditing for network coding storage,” in NetworkCoding (NetCod), 2012 International Symposium on, pp. 155–160, Jun. 2012.

[39] Y. Ishai, E. Kushilevitz, R. Ostrovsky, and A. Sahai, “Batch codes and their applications,” inProceedings of the thirty-sixth annual ACM symposium on Theory of computing, ser. STOC’04, pp. 262–271, 2004.

[40] G. Ateniese, R. Di Pietro, L. V. Mancini, and G. Tsudik, “Scalable and efficient provabledata possession,” in Proceedings of the 4th International Conference on Security andPrivacy in Communication Netowrks, ser. SecureComm ’08. New York, NY, USA: ACMpp. 9:1–9:10, 2008.

[41] R. Dorfman, “The detection of defective members of large populations,” The Annals ofMathematical Statistics, vol. 14, no. 4, pp. 436–440, 1943.

[42] D. Du and F. Hwang, Combinatorial group testing and its applications. World Scientific,1993.

[43] W. Kautz and R. Singleton, “Nonrandom binary superimposed codes,” Information Theory,IEEE Transactions on, vol. 10, no. 4, pp. 363–377, Oct. 1964.

[44] A. Shamir, “How to share a secret,” Commun. ACM, vol. 22, no. 11, pp. 612–613, Nov.1979.

[45] T. Ge and S. Zdonik, “Answering aggregation queries in a secure system model,” in Pro-ceedings of the 33rd international conference on Very large data bases, ser. VLDB ’07.VLDB Endowment, pp. 519–530, 2007.

[46] J. C. Corena and T. Ohtsuki, “A multiple-mac-based protocol to identify misbehaving nodesin network coding,” in Vehicular Technology Conference (VTC Fall), 2012 IEEE, pp. 1–5,Sep. 2012.

[47] J. C. Corena, A. Basu, S. Kiyomoto, and T. Miyake, Y. Ohtsuki, “Xor network codingpollution prevention without homomorphic functions,” in Consumer Communications andNetworking Conference (CCNC), 2014 IEEE, Jan. 2014.

[48] J. Benaloh and M. De Mare, “One-way accumulators: A decentralized alternative to digitalsignatures,” in Advances in CryptologyEUROCRYPT93. Springer, pp. 274–285, 1994.

[49] Y. Dodis, S. Vadhan, and D. Wichs, “Proofs of retrievability via hardness amplification,”in Proceedings of the 6th Theory of Cryptography Conference on Theory of Cryptography,ser. TCC ’09. Berlin, Heidelberg: Springer-Verlag, pp. 109–127, 2009.

[50] D. Du and F. Hwang, Combinatorial group testing and its applications. World Scientific,1993.

REFERENCES 141

[51] A. Le and A. Markopoulou, “Nc-audit: Auditing for network coding storage,” in NetworkCoding (NetCod), 2012 International Symposium on, pp. 155–160, Jun. 2012.

[52] J. Corena and T. Ohtsuki, “Pollution-free regenerating codes with fast reconstruction veri-fication for verifiable cloud storage,” Journal of Networks (To appear), 2014.

[53] J. C. Corena and T. Ohtsuki, “Thwarting diversity attacks in wireless network coding usingthreshold signatures and a sender-centered approach,” in Global Communications Confer-ence (GLOBECOM), 2012 IEEE, pp. 1060–1065, Dec. 2012.

[54] J. Corena and T. Ohtsuki, “Secure and fast aggregation of financial data in cloud-basedexpense tracking applications,” J. Netw. Syst. Manage., vol. 20, no. 4, pp. 534–560, Dec.2012.

[55] J. Dong, R. Curtmola, and C. Nita-Rotaru, “Practical defenses against pollution attacks inwireless network coding,” ACM Trans. Inf. Syst. Secur., vol. 14, pp. 7:1–7:31, Jun. 2011.

[56] R. Blom, “Non-public key distribution.” in Advances in Cryptology: Proceedings ofCRYPTO ’82, pp. 231–236, 1982.

[57] Y. Li, H. Yao, M. Chen, S. Jaggi, and A. Rosen, “Ripple authentication for network coding,”in INFOCOM, 2010 Proceedings, IEEE, pp. 1 –9, Mar. 2010.

[58] R. Canetti, J. Garay, G. Itkis, D. Micciancio, M. Naor, and B. Pinkas, “Multicast security: ataxonomy and some efficient constructions,” in INFOCOM, ’99 Proceedings, IEEE, vol. 2,pp. 708 –716 vol.2, Mar. 1999.

[59] L. Keller, E. Drinea, and C. Pragouli, “Online broadcasting with network coding,” in Net-work Coding, Theory and Applications, 2008. NetCod 2008. Fourth Workshop on. IEEE,pp. 1–6, 2008.

[60] C. Bron and J. Kerbosch, “Algorithm 457: finding all cliques of an undirected graph,”Communications of the ACM, vol. 16, no. 9, pp. 575–577, 1973.

[61] R. L. Rivest, A. Shamir, and L. Adleman, “A method for obtaining digital signatures andpublic-key cryptosystems,” Commun. ACM, vol. 26, pp. 96–99, Jan. 1983.

[62] A. Perrig, R. Canetti, J. D. Tygar, and D. Song, “The tesla broadcast authentication proto-col,” RSA CryptoBytes, pp. 2–13, 2002.

[63] M. Bellare, R. Canetti, and H. Krawczyk, “Keying hash functions for message authentica-tion,” in Proceedings of the 16th Annual International Cryptology Conference on Advancesin Cryptology, ser. CRYPTO ’96. London, UK: Springer-Verlag, pp. 1–15, 1996.

[64] A. Perrig, R. Canetti, D. Song, and J. D. Tygar, “Efficient and secure source authenticationfor multicast,” in In Network and Distributed System Security Symposium, NDSS ’01, pp.35–46, 2001.

[65] C. K.Wong and S. Lam, “Digital signatures for flows and multicasts,” in Network Protocols,1998. Proceedings. Sixth International Conference on, pp. 198 –209, Oct. 1998.

142 REFERENCES

[66] N. NIST, “FIPS 180-2, Secure Hash Standard, Federal Information Processing Standard(FIPS), Publication 180-2,” DEPARTMENTOF COMMERCE, Tech. Rep., Aug. 2002.

[67] New proofs for NMAC and HMAC: Security without collision-resistance, ser. Lecture Notesin Computer Science, vol. 4117. Springer, 2006.

[68] N. Fazio and A. Nicolosi, “Cryptographic accumulators: Definitions, constructionsand applications,” Paper written for course at New York University: www. cs. nyu.edu/nicolosi/papers/accumulators. pdf, 2002.

[69] B. H. Bloom, “Space/time trade-offs in hash coding with allowable errors,” Communica-tions of the ACM, vol. 13, no. 7, pp. 422–426, 1970.

[70] K. Nyberg, “Fast accumulated hashing,” in Fast Software Encryption. Springer, pp. 83–87,1996.

[71] C. Karlof, N. Sastry, Y. Li, A. Perrig, and J. Tygar, “Distillation codes and applications todos resistant multicast authentication,” in Proceedings of the ISOC Symposium on Networkand Distributed System Security (SNDSS). Citeseer, pp. 37–56, 2004.

[72] Z. Yu, Y. Wei, B. Ramkumar, and Y. Guan, “An efficient scheme for securing xor networkcoding against pollution attacks,” in INFOCOM 2009, Proceedings. IEEE, pp. 406–414,2009.

[73] J. M. Park, E. K. P. Chong, and H. J. Siegel, “Efficient multicast packet authenticationusing signature amortization,” in 2002 IEEE Symposium on Security and Privacy. IEEEComputer Society, 2002.

[74] R. Rivest, “The md5 message-digest algorithm,” 1992.

[75] D. Boneh and D. M. Freeman, “Homomorphic signatures for polynomial functions,” inAdvances in Cryptology–EUROCRYPT 2011. Springer, pp. 149–168, 2011.

[76] M. O. Rabin, “Efficient dispersal of information for security, load balancing, and fault tol-erance,” J. ACM, vol. 36, no. 2, pp. 335–348, 1989.

[77] N. Cao, S. Yu, Z. Yang, W. Lou, and Y. T. Hou, “Lt codes-based secure and reliable cloudstorage service,” in INFOCOM, 2012 Proceedings, IEEE, pp. 693–701, 2012.

[78] H. C. H. Chen and P. P. C. Lee, “Enabling data integrity protection in regenerating-coding-based cloud storage,” in 31st IEEE International Symposium on Reliable Distributed Sys-tems (SRDS 2012), 2012.

[79] P. Damaschke and A. S. Muhammad, “Bounds for nonadaptive group tests to estimate theamount of defectives,” in Combinatorial Optimization and Applications. Springer, pp.117–130, 2010.

[80] R. Kumar, S. Rajagopalan, and A. Saha, “Coding constructions for blacklisting problemswithout computational assumptions,” in Advances in Cryptology CRYPTO 99, ser. LectureNotes in Computer Science. Springer, pp. 783–783, 1999.

REFERENCES 143

[81] K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Enabling node repair in any erasure code fordistributed storage,” in ISIT, A. Kuleshov, V. Blinovsky, and A. Ephremides, Eds. IEEE,pp. 1235–1239, 2011.

[82] Google, “Google play - google+ - traveling this season and want to make sure your musicgoes (retrieved may 10th 2014),” [Online] available:https://plus.google.com/+GooglePlay/posts/VZhB6EpsWKx, 2013.

[83] V. Chandrasekhar, M. Sharifi, and D. A. Ross, “Survey and evaluation of audio fingerprint-ing schemes for mobile query-by-example applications.” in ISMIR, vol. 20, pp. 801–806,2011.

[84] D.-Z. Du, G.-L. Xue, S.-Z. Sun, and S.-W. Cheng, “Modifications of competitive grouptesting,” SIAM Journal on Computing, vol. 23, no. 1, pp. 82–96, 1994.

[85] A. Martonik, “Google music scan and match only adding clean versions of songs,”[Online] available: http://www.androidcentral.com/google-music-scan-and-match-only-adding-clean-versions-songs (Retrieved May 14th 2014), 2012.

[86] G. Liang, R. Agarwal, and N. Vaidya, “When watchdog meets coding,” in INFOCOM 2010,Proceedings, IEEE, pp. 2267–2275, 2010.

[87] P. Gallagher, D. D. Foreword, and C. F. Director, “Fips pub 186-3 federal informationprocessing standards publication digital signature standard (dss),” 2009.

[88] V. Shoup, “Practical threshold signatures.” in Advances in Cryptology - EUROCRYPT 2000,Proceedings, 2000, pp. 207–220.

[89] D. Boneh, X. Ding, G. Tsudik, and C. M. Wong, “Amethod for fast revocation of public keycertificates and security capabilities,” in 10th conference on USENIX Security Symposium,ser. SSYM’01, 2001.

[90] D. Boneh, “Twenty years of attacks on the rsa cryptosystem,” NOTICES OF THE AMS,vol. 46, pp. 203–213, 1999.

[91] M.-S. Hwang and K.-F. Hwang, “Cryptanalysis of the batch verifying multiple rsa digitalsignatures,” Informatica, vol. 11, no. 1, pp. 15–19, 2000.

[92] E. Mykletun, M. Narasimha, and G. Tsudik, “Authentication and integrity in outsourceddatabases,” ACM Transactions on Storage (TOS), vol. 2, no. 2, pp. 107–138, 2006.

[93] D. Boneh, C. Gentry, B. Lynn, and H. Shacham, “Aggregate and verifiably encrypted sig-natures from bilinear maps,” in Advances in cryptologyEUROCRYPT 2003. Springer, pp.416–432, 2003.

[94] W. Diffie and M. E. Hellman, “New directions in cryptography,” Information Theory, IEEETransactions on, vol. 22, no. 6, pp. 644–654, 1976.

[95] D. Boneh, “The decision diffie-hellman problem,” in Algorithmic number theory. Springer,pp. 48–63, 1998.

144 REFERENCES

[96] J. Katz and A. Y. Lindell, “Aggregate message authentication codes,” in CT-RSA’08, ser.CT-RSA’08, pp. 155–169, 2008.

[97] K. M. Martin, J. Pieprzyk, R. Safavi-Naini, H. Wang, and P. R. Wild, “Threshold macs,”in Proceedings of the 5th international conference on Information security and cryptology,ser. ICISC’02. Berlin, Heidelberg: Springer-Verlag, pp. 237–252, 2003.

[98] D. Boneh, B. Lynn, and H. Shacham, “Short signatures from the weil pairing,” J. Cryptol-ogy, vol. 17, no. 4, pp. 297–319, 2004.

[99] D. Boneh, C. Gentry, B. Lynn, and H. Shacham, “A survey of two signature aggregationtechniques,” 2003.

[100] B. Lynn, “Pairing-based cryptography library,” hhttp://crypto.stanford.edu/pbc/, 2006.

[101] ——, “Type a internals - pbc library manual,” http://crypto.stanford.edu/pbc/manual.pdf,pp. 37–37, 2006.

[102] Amazon Web Services, “Amazon web services customer agreement,” 2011. [Online].Available: http://aws.amazon.com/agreement/#10

[103] QuickReceipts, “Quickreceipts - your easy online receipt manager,” 2011. [Online].Available: http://myquickreceipts.intuit.com/

[104] Alletronic, “Alletronic - your paperless convienence,” 2011. [Online]. Available:http://www.alletronic.com/

[105] S. Kamara and K. Lauter, “Cryptographic cloud storage,” in Proceedings of the 14th in-ternational conference on Financial cryptograpy and data security, ser. FC’10. Berlin,Heidelberg: Springer-Verlag, pp. 136–149, 2010.

[106] R. Agrawal, J. Kiernan, R. Srikant, and Y. Xu, “Order preserving encryption for numericdata,” in Proceedings of the 2004 ACM SIGMOD international conference on Managementof data, ser. SIGMOD ’04. New York, NY, USA: ACM, pp. 563–574, 2004.

[107] A. Boldyreva, N. Chenette, Y. Lee, and A. O’Neill, “Order-preserving symmetric encryp-tion,” in Advances in Cryptology - EUROCRYPT 2009, ser. Lecture Notes in ComputerScience, A. Joux, Ed. Springer Berlin / Heidelberg, vol. 5479, pp. 224–241, 2009.

[108] M. Bellare, A. Boldyreva, and A. O’Neill, “Deterministic and efficiently searchableencryption,” in Proceedings of the 27th annual international cryptology conference onAdvances in cryptology, ser. CRYPTO’07. Berlin, Heidelberg: Springer-Verlag, pp.535–552, 2007.

[109] A. Boldyreva, N. Chenette, Y. Lee, and A. Oneill, “Order-preserving symmetric encryp-tion,” in Advances in Cryptology-EUROCRYPT 2009. Springer, pp. 224–241, 2009

REFERENCES 145

[110] D. Belazzougui, P. Boldi, R. Pagh, and S. Vigna, “Monotone minimal perfect hashing:searching a sorted table with o (1) accesses,” in Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics,pp. 785–794, 2009.

[111] Q. Tang, “Privacy preserving mapping schemes supporting comparison,” in Proceedings ofthe 2010 ACM workshop on Cloud computing security workshop. ACM, pp. 53–58, 2010.

[112] H. Hacıgumus, B. Iyer, and S. Mehrotra, “Efficient execution of aggregation queries overencrypted relational databases,” inDatabase Systems for Advanced Applications. Springer,pp. 125–136, 2004.

[113] R. A. Popa, C. M. S. Redfield, N. Zeldovich, and H. Balakrishnan, “Cryptdb: protectingconfidentiality with encrypted query processing,” in Proceedings of the Twenty-Third ACMSymposium on Operating Systems Principles, ser. SOSP ’11. New York, NY, USA: ACM,pp. 85–100, 2011.

[114] D. X. Song, D. Wagner, and A. Perrig, “Practical techniques for searches on encrypteddata,” in Security and Privacy, 2000. S&P 2000. Proceedings. 2000 IEEE Symposium on.IEEE, pp. 44–55, 2000.

[115] T. P. Pedersen, “Non-interactive and information-theoretic secure verifiable secret sharing,”in Advances in CryptologyCRYPTO91. Springer, pp. 129–140, 1992.

[116] S. Lucks, “The sum of prps is a secure prf,” in Advances in Cryptology EUROCRYPT 2000,ser. Lecture Notes in Computer Science. Springer Berlin / Heidelberg, vol. 1807, pp.470–484, 2000.

[117] M. Bellare, A. Desai, E. Jokipii, and P. Rogaway, “A concrete security treatment of sym-metric encryption,” Foundations of Computer Science, Annual IEEE Symposium on, vol. 0,pp. 394, 1997.

[118] A. Rukhin, J. Soto, J. Nechvatal, M. Smid, E. Barker, S. Leigh, L. Mark, D. Vangel,Mark Banks, H. Alan, J. Dray, and S. Vo, “A statistical test suite for random and pseu-dorandom number generators for cryptographic applications,” NIST-National Institute ofStandards and Technology, Tech. Rep., April 2010.

Improvements on aggregation and security of encoded data for … · 2020-02-05 · Improvements on...

Documents

Transcript of Improvements on aggregation and security of encoded data for … · 2020-02-05 · Improvements on...