Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS:...
Transcript of Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS:...
![Page 1: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/1.jpg)
Security and Data Privacy
Instructor: Pratiksha Thakercs245.stanford.edu
![Page 2: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/2.jpg)
Outline
Security requirements
Key concepts and tools
Differential privacy
Other security tools
CS 245 2
![Page 3: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/3.jpg)
Outline
Security requirements
Key concepts and tools
Differential privacy
Other security tools
CS 245 3
![Page 4: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/4.jpg)
Why Security & Privacy?
CS 245 4
![Page 5: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/5.jpg)
Why Security & Privacy?
Data is valuable & can cause harm if released» Example: medical records, purchase history,
internal company documents, etc
Data releases can’t usually be “undone”
Security policies can be complex» Each user can only see data from their friends» Analyst can only query aggregate data» Users can ask to delete their derived data
CS 245 5
![Page 6: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/6.jpg)
Why Security & Privacy?
It’s the law! New regulations about user data:
US HIPAA: Health Insurance Portability & Accountability Act (1996)
» Mandatory encryption, access control, training
EU GDPR, CA CCPA: (2018)» Users can ask to see & delete their data
PCI DSS: Payment Card Industry standard (2004)
» Required in contracts with MasterCard, etcCS 245 6
![Page 7: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/7.jpg)
Consequence
Security and privacy must be baked into the design of data-intensive systems
» Often a key differentiator for products!
CS 245 7
![Page 8: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/8.jpg)
The Good News
Declarative interface to many data-intensive systems can enable powerful security features
» One of the “big ideas” in our class!
Example: System R’s access control on views
CS 245 8
arbitrarySQL query
TablesSQLView Users
read
write
![Page 9: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/9.jpg)
Outline
Security requirements
Key concepts and tools
Differential privacy
Other security tools
CS 245 9
![Page 10: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/10.jpg)
Some Security Goals
Access Control: only the “right” users can perform various operations; typically relies on:
» Authentication: a way to verify user identity (e.g. password)
» Authorization: a way to specify what users may take what actions (e.g. file permissions)
Auditing: system records an incorruptible audit trail of who did each action
CS 245 10
![Page 11: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/11.jpg)
Some Security Goals
Confidentiality: data is inaccessible to external parties (often via cryptography)
Integrity: data can’t be modified by external parties
Privacy: only a limited amount of information about “individual” users can be learned
CS 245 11
![Page 12: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/12.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?
12
![Page 13: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/13.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades
13
![Page 14: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/14.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades2. Bobby sending hand-crafted network packets to
Axess to change his grades
14
![Page 15: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/15.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades2. Bobby sending hand-crafted network packets to
Axess to change his grades3. Bobby getting a job as a DB admin at Axess
15
![Page 16: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/16.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades2. Bobby sending hand-crafted network packets to
Axess to change his grades3. Bobby getting a job as a DB admin at Axess4. Bobby guessing Matei’s password
16
![Page 17: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/17.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades2. Bobby sending hand-crafted network packets to
Axess to change his grades3. Bobby getting a job as a DB admin at Axess4. Bobby guessing Matei’s password5. Bobby blackmailing Matei to change his grade
17
![Page 18: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/18.jpg)
Clarifying These GoalsSay our goal was access control: only Matei can set CS 245 student grades on Axess
What scenarios should Axess protect against?1. Bobby T. (an evil student) logging into Axess as
himself and being able to change grades2. Bobby sending hand-crafted network packets to
Axess to change his grades3. Bobby getting a job as a DB admin at Axess4. Bobby guessing Matei’s password5. Bobby blackmailing Matei to change his grade6. Bobby discovering a flaw in AES to do #2
18
![Page 19: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/19.jpg)
Threat Models
To meaningfully reason about security, need a threat model: what adversaries may do
» Same idea as failure models!
For example, in our Axess scenario, assume:» Adversaries only interact with Axess through
its public API» No crypto algorithm or software bugs» No password theft
CS 245 19
Implementing complex security policies can be hard even with these assumptions!
![Page 20: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/20.jpg)
Threat Models
No useful threat model can cover everything» Goal is to cover the most feasible scenarios
for adversaries to increase the cost of attacks
Threat models also let us divide security tasks across different components
» E.g. auth system handles passwords, 2FA
CS 245 20
![Page 21: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/21.jpg)
Threat Models
CS 245 21Source: XKCD.com
![Page 22: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/22.jpg)
Useful Building Blocks
Encryption: encode data so that only parties with a key can efficiently decrypt
Cryptographic hash functions: hard to find items with a given hash (or collisions)
Secure channels (e.g. TLS): confidential, authenticated communication for 2 parties
CS 245 22
![Page 23: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/23.jpg)
Security Tools from DBMSs
First-class concept of users + access control» Views as in System R, tables, etc
Secure channels for network communication
Audit logs for analysis
Encrypt data on-disk (perhaps at OS level)
CS 245 23
![Page 24: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/24.jpg)
Modern Tools for Security
Privacy metrics and enforcement thereof(e.g. differential privacy)
Computing on encrypted data (e.g. CryptDB)
Hardware-assisted security (e.g. enclaves)
Multi-party computation (e.g. secret sharing)
CS 245 24
![Page 25: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/25.jpg)
Outline
Security requirements
Key concepts and tools
Differential privacy
Other security tools
CS 245 25
![Page 26: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/26.jpg)
Threat Model
CS 245 26
Table withprivate data Data
analysts
Databaseserver
• Database software is working correctly• Adversaries only access it through public API• Adversaries have limited # of user accounts
queries
queries
![Page 27: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/27.jpg)
Private statistics
27
SELECT AVG(income) FROM professors WHERE state=“California”
![Page 28: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/28.jpg)
Private statistics
28
Are aggregate statistics more private than individual data?
SELECT AVG(income) FROM professors WHERE state=“California”
SELECT AVG(income) FROM professors WHERE name=“Matei Zaharia”
![Page 29: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/29.jpg)
Private statistics
29
Are aggregate statistics more private than individual data? No!
SELECT AVG(income) FROM professors WHERE state=“California”
SELECT AVG(income) FROM professors WHERE name=“Matei Zaharia”
![Page 30: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/30.jpg)
Private statistics
30
![Page 31: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/31.jpg)
Private statistics
31
![Page 32: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/32.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
32
![Page 33: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/33.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
Intuition: the function is differentially private if removing or changing a data point does not change the output "too much"
33
![Page 34: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/34.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
Intuition: the function is differentially private if removing or changing a data point does not change the output "too much"
Intuition: plausible deniability
34
![Page 35: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/35.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
For A and B that differ in one element,
35
![Page 36: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/36.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
For A and B that differ in one element,
36
Randomized algorithm that computes statistic
![Page 37: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/37.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
For A and B that differ in one element,
37
Any subset of possible outcomes
![Page 38: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/38.jpg)
Idea: differential privacy
A contract for algorithms that output statistics
For A and B that differ in one element,
38
Privacy parameterSmaller ε ~= more privacy, less accuracy
![Page 39: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/39.jpg)
Similar outcomes whether or not Matei in DB
What Does It Mean?
Say an adversary runs some query and observes a result
Adversary had some set of results, S, that lets them infer something about Matei if
Then:
CS 245 39
≈ 1+ε
![Page 40: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/40.jpg)
What Does It Mean?
CS 245 40
Private information is noisy. Can we determine anything useful?
![Page 41: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/41.jpg)
What Does It Mean?
CS 245 41
Private information is noisy. Can we determine anything useful?
Query:
SELECT COUNT(*) FROM patients WHERE causeOfDeath = ...
Assume ε=0.1
![Page 42: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/42.jpg)
What Does It Mean?
CS 245 42
![Page 43: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/43.jpg)
What Does It Mean?
CS 245 43
![Page 44: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/44.jpg)
What Does It Mean?
CS 245 44
With enough base signal, DP can still give useful
information!
![Page 45: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/45.jpg)
Side Information
Consider the following query:
SELECT AVG(income) FROM professors WHERE state=“CA”
45
![Page 46: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/46.jpg)
Side Information
Consider the following query:
SELECT AVG(income) FROM professors WHERE state=“CA”
What if adversary knows:"Matei makes $100k more than the average professor in CA"
46
![Page 47: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/47.jpg)
Side Information
Consider the following query:
SELECT AVG(income) FROM professors WHERE state=“CA”
What if adversary knows:"Matei makes $100k more than the average professor in CA"
Equally bad for Matei whether or not he personally participates in DB!
47
![Page 48: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/48.jpg)
Some Nice Properties of Differential PrivacyComposition: can reason about the privacy effect of multiple (even dependent) queries
Let queries Mi each provide εi-differential privacy; then the set of queries {Mi} provides Σiεi-differential privacy
Proof:
CS 245 48
Adversary’s ability to distinguish DBs A & B grows in a bounded way with each query
![Page 49: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/49.jpg)
Some Nice Properties of Differential PrivacyParallel composition: even better bounds if queries are on disjoint subsets (e.g., histograms)
Let Mi each provide ε-differential privacy and read disjoint subsets of the data Di; then the set of queries {Mi} provides ε-differential privacy
Example: query both average patient age in CA and average patient age in NY
CS 245 49
![Page 50: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/50.jpg)
Some Nice Properties of Differential PrivacyEasy to compute: can use known results for various operators, then compose for a query
» Enables systems to automatically compute privacy bounds given declarative queries!
CS 245 50
![Page 51: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/51.jpg)
Disadvantages of Differential Privacy
CS 245 51
![Page 52: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/52.jpg)
Disadvantages of Differential PrivacyEach user can only make a limited number of queries (more precisely, limited total ε)
» Their ε grows with each query and can’t shrink» Have to specify how much ε to use
How to set ε in practice?» Apple system: 2? 4? 16?» NBER paper: 8?» Hard to tell what various values mean, though
there is a nice Bayesian interpretation
CS 245 52
![Page 53: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/53.jpg)
Each user can only make a limited number of queries (more precisely, limited total ε)
» Their ε grows with each query and can’t shrink» Have to specify how much ε to use
How to set ε in practice?» Apple system: 2? 4? 16?» NBER paper: 8?» Hard to tell what various values mean, though
there is a nice Bayesian interpretation
Disadvantages of Differential Privacy
CS 245 53
![Page 54: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/54.jpg)
Computing Differential Privacy BoundsLet’s start with COUNT aggregates:SELECT COUNT(*) FROM A
The randomized algorithm M(A) that returns |A| + Laplace(1/ε) is ε-differentially private
CS 245 54Image source: Wikipedia
Laplace(b) distribution:p(x) = 1/(2b) e-|x|/b
Mean: 0Variance: 2b2
![Page 55: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/55.jpg)
Computing Differential Privacy BoundsLet’s start with COUNT aggregates:SELECT COUNT(*) FROM A
The randomized algorithm M(A) that returns |A| + Laplace(1/ε) is ε-differentially private
CS 245 55
Result of M(A)for count(A)=107
Value returned by M
Result of M(B)for count(B)=108
Pro
babi
lity
![Page 56: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/56.jpg)
Computing Differential Privacy BoundsWhat about AVERAGE aggregates:SELECT AVERAGE(x) FROM A
CS 245 56
![Page 57: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/57.jpg)
Computing Differential Privacy BoundsWhat about AVERAGE aggregates:SELECT AVERAGE(x) FROM A
How much can one element of A affect result?» In general case, unboundedly much! No privacy
•SELECT AVG(wealth) WHERE city=“Omaha, NE”
» If x ∈ [0,m] for all x in A, then by at most m•Adding Laplace(m/ε) noise is ε-differentially private
Paper bounds AVG, SUM for values x ∈ [-1,1]CS 245 57
![Page 58: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/58.jpg)
Computing Differential Privacy BoundsGeneral notion to capture the impact of one element: sensitivity
Sensitivity of a function f: U→ℝ on sets is
Δf = maxA,B∈U differ in 1 element |f(A) – f(B)|
CS 245 58
![Page 59: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/59.jpg)
Sensitivity Examples
f(A) = |A|
f(A) = sum(A), x∈[0,m] ∀x∈A
f(A) = avg(A), x∈[0,m] ∀x∈A
f(A) = |{x∈A | x is male}|
f(A) = |A⨝B|
f(A) = |A⨝B|, each key has ≤ k matches
CS 245 59
1
m
m
1
unbounded
k
Sensitivity
![Page 60: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/60.jpg)
Multi-dimensional Sensitivity
Can also define sensitivity for functions that return multiple numerical results:
Sensitivity of a function f: U→ℝd on sets is
Δf = maxA,B∈U differ in 1 element ||f(A) – f(B)||1
Example: f fits a linear model to the data...
CS 245 60
![Page 61: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/61.jpg)
Computing Differential Privacy Bounds
How do we reason about functions involving cardinality (e.g. GROUP BY...)?
CS 245 61
![Page 62: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/62.jpg)
Computing Differential Privacy Bounds
How do we reason about functions involving cardinality (e.g. GROUP BY...)?
Concept introduced in PINQ: stability
CS 245 62
![Page 63: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/63.jpg)
Computing Differential Privacy BoundsConcept introduced in PINQ: stability
A function T on sets is c-stable if for any two input sets A and B,
|T(A) ⊕ T(B)| ≤ c |A ⊕ B|
CS 245 63
Number of recordsthat differ in A and B
PINQ’s approach: let user do any # of set ops; compute their stability; then let them do one
aggregate op and compute its sensitivity
![Page 64: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/64.jpg)
Stability Examples
CS 245 64
T(A) = σpredicate(A) (“Where”)
T(A) = πexprs(A) (“Select”)
T(A, B) = A ∪ B
T(A) = GroupBy(A, expr) (retruns 1 record/group)
T(A) = A⨝B limited to at most 1 match per key
1
1
1
2
1
Stability
![Page 65: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/65.jpg)
Partition Operator
Partition(dataset, key_list) returns a set of IQueryables: one for each key in your list
» User provides the desired keys in advance (e.g. “CA” or “NY”); can’t use to discover keys
» Lets PINQ use parallel composition rule since the sets returned are all disjoint
Stability = 1
CS 245 65
![Page 66: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/66.jpg)
Analyzing Queries in PINQ
User calls multiple set transformation ops and finally one aggregation/result op
» Transformations are lazy; can’t see result
PINQ computes stability of set ops and multiplies by sensitivity of each aggregate to get total sensitivity
User provides an ε to aggregate; PINQ adds noise proportional to sensitivity/ε
CS 245 66
![Page 67: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/67.jpg)
Putting It All Together
CS 245 67
cricket: 127123.313
![Page 68: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/68.jpg)
Putting It All Together
CS 245 68
![Page 69: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/69.jpg)
Uses of Differential Privacy
Statistics collection about iOS features
“Randomized response”: clients add noise to data they send instead of relying on provider
Research systems that use DP to measure security (e.g. Vuvuzela messaging)CS 245 69
queriesx
bob + noise
xalice + noise
![Page 70: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/70.jpg)
Outline
Security requirements
Key concepts and tools
Differential privacy
Other security tools
CS 245 70
![Page 71: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/71.jpg)
Computing on Encrypted Data
Threat model: adversary has access to the database server we run on (e.g. in cloud)
Idea: some encryption schemes allow computing on data without decrypting it:
fenc(Enc(X)) = Enc(f(X))
Usually very expensive, but can be done efficiently for some functions f!
CS 245 71
![Page 72: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/72.jpg)
Example Systems
CryptDB, Mylar (MIT research projects)
Encrypted BigQuery (CryptDB on BigQuery)
Leverage properties of SQL to come up with efficient encryption schemes & query plans
CS 245 72
![Page 73: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/73.jpg)
Example Schemes
Equality checks with deterministic encryption
SELECT * FROM table WHERE state=“CA”
SELECT * FROM table WHERE state=“XAYDS9”
CS 245 73
Encrypt “state” column
![Page 74: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/74.jpg)
Example Schemes
Equality checks with deterministic encryption
SELECT * FROM table WHERE state=“CA”
SELECT * FROM table WHERE state=“XAYDS9”
Potential challenges with this scheme:» Adversary can see relative frequency of keys» Adversary sees which keys are accessed on
each query (e.g. Matei logs in → CA key read)CS 245 74
Encrypt “state” column
![Page 75: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/75.jpg)
Other Encryption Schemes
Additive homomorphic encryption:
Enc(A + B) = Enc(A) ⍟ Enc(B)
Fully homomorphic encryption:
Enc(f(A)) = fenc(Enc(A))
Order-preserving encryption:
if A < B then Enc(A) < Enc(B)CS 245 75
Possible but very expensive(108 or more overhead)
![Page 76: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/76.jpg)
Hardware Enclaves
Threat model: adversary has access to the database server we run on (e.g. in cloud) but can’t tamper with hardware
Idea: CPU provides an “enclave” that can provably run some code isolated from the OS
» Enclaves returns a certificate signed by CPU maker that it ran code C on argument A
CS 245 76
![Page 77: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/77.jpg)
Hardware Enclaves in Practice
Already present in all Intel CPUs (Intel SGX), and many Apple custom chips (T2, etc)
Initial applications were digital rights mgmt., secure boot, secure login
» Protect even against a compromised OS
Some research systems explored using these for data analytics: Opaque, ObliDB, others
CS 245 77
![Page 78: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/78.jpg)
Databases + Enclaves
1. Store data encrypted with an encryption scheme that leaks nothing (randomized)
2. With each query, user includes a public key kq to encrypt the result with
3. Database runs a function f in the enclave that does query and encrypts result with kq
4. User can verify f ran, DB can’t see result!
CS 245 78Performance is fast too (normal CPU speed)!
![Page 79: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/79.jpg)
Are Enclaves Enough to Secure Against Non-HW Adversaries?
CS 245 79
![Page 80: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/80.jpg)
Are Enclaves Enough to Secure Against Non-HW Adversaries?Not quite! adversary can still learn info by observing access patterns to RAM or timing
» Similar to some attacks on encrypted DBs
Oblivious algorithms can help prevent this but add more computational cost
» Oblivious = same access pattern regardless of underlying data, query result, etc
CS 245 80
![Page 81: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/81.jpg)
Multi-Party Computation (MPC)
Threat model: participants p1, …, pn want to compute some joint function f of their data but don’t trust each other
» E.g. patient stats across 2 hospitals
Idea: protocols that compute f without revealing anything else to participants
» Like with encryption, general computations are possible but expensive
CS 245 81
![Page 82: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/82.jpg)
Example: Secret Sharing
Users wants to store a secret value x among n servers, but doesn’t fully trust them
» E.g. the servers are public clouds… what if one gets hacked?
Idea: split x into “shares” xi so that all shares are needed to recover x
Additive secret sharing: x = integer mod P,xi are random integers so Σxi = x
CS 245 82
![Page 83: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/83.jpg)
Secret Sharing Example
CS 245 83
Userx = 5 (mod 10)
x1 = 3(mod 10)
x2 = 8(mod 10)
x2 = 4(mod 10)
Servers
3 + 8 + 4 = 5 (mod 10)
?? ??
Note: performance is quite fast (just additions)
![Page 84: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/84.jpg)
Function Secret Sharing
Recent result that allows sharing some functions too (keeping queries private)
Splinter (optional paper): uses FSS to run privateSQL queries on publicdata like Google Maps
CS 245 84
Splinter Server Library
Splinter Server Library
Splinter Server Library
Servers
Splinter Client
User
Parametrized query: SELECT TOP 10 restaurant WHERE city = ? AND cuisine = ? ORDER BY rating
private parameters
![Page 85: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/85.jpg)
Lineage Tracking and RetractionGoal: keep track of which data records were derived from an individual input record
» Facilitate removing a user’s data in GDPR, verifying compliance, etc
Some real systems provide this already at low granularity, but could be baked into DB
CS 245 85
![Page 86: Security and Dataweb.stanford.edu/class/cs245/slides/16-Security-and-Privacy.pdf · PCI DSS: Payment Card Industry standard (2004) »Required in contracts with MasterCard, etc CS](https://reader033.fdocuments.us/reader033/viewer/2022060509/5f256c684cd48b72fc6b2535/html5/thumbnails/86.jpg)
Summary
Security and data privacy are essential concerns for data-intensive systems
Threat models are a systematic way to measure security and reason about designs
Many nice theoretical tools exist to reason about security needs of relational & math ops
» Build on declarative and relational APIs!
CS 245 86