Post on 22-Dec-2015
Future Privacy and Security Problem
• Real-time location information– Friend Finder (“where is Alice?”)
– Filtered searches (“restaurants near me?”)
– Better awareness (“Daniel is at school”)
• What kinds of controls and feedback needed?
Find Friends inTouch
Future Privacy and Security Problem
• You think you are in one context, actually overlapped in many others
• Without this understanding, cannot act appropriately
Usable Privacy and Security Important
• People increasingly asked to make trust judgements– Install this software?
– Login to a site and enter username and password?
– Share location information?
– What context you are in, how to act?
• New networked technologies leading to new risks
Everyday Risks Extreme Risks
Hackers, Muggers_________________________________
Identity TheftMalware
Personal safety
Employers_________________________________
Over-monitoringDiscrimination
Reputation
Friends, Family_________________________________
Over-protectionSocial obligationsEmbarrassment
Government__________________________
Civil liberties
Grand Challenge
“Give end-users security controls they can understandand privacy they can control for the dynamic, pervasive computing environments of the future.”
- Computing Research Association 2003
Usable Privacy and Security Work
SupportingTrust Decisions
UbiquitousComputing
LocationEnhancedServices
Project: Supporting Trust Decisions
• Goal here is to help people make better decisions– Context here is anti-phishing
• Large multi-disciplinary team project– Six faculty, five PhD students
Phishing
• A semantic attack aimed directly at people rather than computers– “Please update your account”
– “Fill out survey and get $25”
– “Question about your auction”
• Rapidly growing in scale and damage– ~7000 new phishing sites in Dec 2005 alone
– ~$1 billion in damages
– More profitable (and safer) to phish than rob a bank
Outline
• Human-Side of Anti-Phishing– Interviews to understand decision-making
– Embedded Training
– Anti-Phishing Game
• Computer-Side– Email Anti-Phishing Filter
– Automated Testbed for Anti-Phishing Toolbars
– Our Anti-Phishing Toolbar
• Automate where possible, support where necessary
Project: Supporting Trust DecisionsInterviews to Understand Decision-Making
• How do people decide what e-mails to “trust”?
• Interviews with 40 novices and some experts– Asked them to role play and go through a series of emails
Project: Supporting Trust DecisionsInterviews to Understand Decision-Making
• How do people decide what e-mails to “trust”?
• Interviews with 40 novices and some experts– Asked them to role play and go through a series of emails
• Highlights– People know cues (from, to, locks) but interpret incorrectly
• Very few people understand URLs• Browser chrome versus content
– Hard for people to generalize risks (Banks vs. Amazon)
– Judge legitimacy primarily by quality of site
– Was expecting an email or have had previous contact
Outline
• Human-Side of Anti-Phishing– Interviews to understand decision-making
– Embedded Training
– Anti-Phishing Game
• Computer-Side– Email Anti-Phishing Filter
– Testbed for Anti-Phishing Toolbars
– Our Anti-Phishing Toolbar
Project: Supporting Trust DecisionsEmbedded Training
• Can we “train” people to avoid phishing in their regular use of email?– Periodically, people get sent a training email
– Training email looks like a phishing attack
– If person falls for it, intervention warns and highlights what cues to look for
• Has been done by others– New York state government office, West Point, Indiana U
• Goal: Understand what designs are most effective
Project: Supporting Trust DecisionsEmbedded Training
• Created three interventions– #0 – Early prototype that helped us explore design space
– #1 – Diagram that explains phishing
– #2 – Comic strip that tells a story
– Shown only if a person clicks on a link in email
#0 – Early Prototype•People didn’t understand what the training message was trying to say
• Why am I getting this?• Missed explanation text at top
•Screenshot of the web browser confused people
•People who clicked on a phishing link were very likely to enter in username and password
•Need clear actionable items• Not the same, so what?
Embedded Training Evaluation
• Compared two prototypes to standard security notices– A – EBay, PayPal notices
– B – Diagram that explains phishing
– C – Comic strip that tells a story
• 10 participants in each condition (30 total)• Roughly, go through 19 emails, 4 phishing attacks
scattered throughout, 2 training emails too– Emails are in context of working in an office
Embedded Training Results
0102030405060708090
100
Emails which had links in them
Pe
rce
nta
ge
of
use
rs w
ho
clic
ke
d
on
a li
nk
Group A Group B Group C
Embedded Training Summary
• Summary– Existing practice of security notices ineffective
– Diagram intervention mildly better
– Comic strip intervention worked best
• Next Steps– Iterate on the design
– Understand more why comic strip worked better• Story? Comic format?
– Larger scale deployment and evaluation
Anti-Phishing Phil
• A game to teach people about anti-phishing– Embedded training focuses on email
– Game focuses on web browser, urls
• Goals– How to parse URLs
– Where to look for URLs
– Use search engines instead
• Early preview!
Outline
• Human-Side of Anti-Phishing– Interviews to understand decision-making
– Embedded Training
– Anti-Phishing Game
• Computer-Side– Email Anti-Phishing Filter
– Testbed for Anti-Phishing Toolbars
– Our Anti-Phishing Toolbar
Email Anti-Phishing Filter
• Philosophy: automate where possible, support where necessary
• Goal: Create an email filter that detects phishing emails– Well explored area for spam
– Can we do better for phishing?
Email Anti-Phishing Filter
• Heuristics combined in SVM– IP addresses in links (http://128.23.34.45/blah)
– Age of linked-to domains (younger domains likely phishing)
– Non-matching URLs (ex. most links point to PayPal)
– “Click here to restore your account”
– HTML email
– Number of links
– Number of domain names in links
– Number of dots in URLs (http://www.paypal.update.example.com/update.cgi)
– JavaScript
– SpamAssassin rating
Email Anti-Phishing Filter Evaluation
• Ham corpora from SpamAssassin (2002 and 2003)– 6950 good emails
• Phishingcorpus– 860 phishing emails
Outline
• Human-Side of Anti-Phishing– Interviews to understand decision-making
– Embedded Training
– Anti-Phishing Game
• Computer-Side– Email Anti-Phishing Filter
– Testbed for Anti-Phishing Toolbars
– Our Anti-Phishing Toolbar
Testbed for Anti-Phishing Toolbars
• Lots of anti-phishing web browser toolbars, but unclear how well they work in practice– Way of systematically evaluating toolbars
– Way of rigorously comparing algorithms
Testbed for Anti-Phishing Toolbars
• First iteration: manual evaluation– Get 1 laptop and 1 person per toolbar
– Send out a URL
– Manually check
– Tedious, slow, error-prone
• Created a testbed that could semi-automatically evaluate these toolbars– Just give it a set of URLs to check (labeled as phish or not)
– Check all the toolbars, aggregate statistics
Testbed for Anti-Phishing Toolbars
• Two key systems issues
• #1 – How to get a list of phishing URLs to evaluate?– Phishing feed from Anti-Phishing Working Group (APWG)
– Manually inspect each URL to confirm phish
• #2 – How to automate this for different toolbars?– Different APIs (if any), different browsers
– Image-based approach, take screenshots of web browser and compare relevant portions to known states
Evaluation
• Tested five toolbars– NetCraft v1.6.2
– TrustWatch v3.0.4.0.1.2
– SpoofGuard (uses heuristics only)
– CloudMark v1.0
– Google Toolbar v2.1
• Test URLs manually confirmed– Extracted 100 confirmed, active phishing URLs
spanning 100 domains
– Also extracted 60 legitimate domains and added 40 others (banks, etc)
Results
Accuracy
0. 0%
20. 0%
40. 0%
60. 0%
80. 0%
100. 0%
0 1 2 12 24Ti me
Accu
racy spoofguard
trustwatchgoogl ecl oudmarknetcraf t
Results
• Stanford’s SpoofGuard and NetCraft had best results• CloudMark was worst
– Relies on user ratings, perhaps not updated fast enough?
• Stanford’s SpoofGuard only one with false positives
Outline
• Human-Side of Anti-Phishing– Interviews to understand decision-making
– Embedded Training
– Anti-Phishing Game
• Computer-Side– Email Anti-Phishing Filter
– Testbed for Anti-Phishing Toolbars
– Our Anti-Phishing Toolbar
Our Anti-Phishing Toolbar
• Issue #1: can we do better in detecting phish?– SpoofGuard accuracy 90-95%, but lots of false positives
– NetCraft also around 90-95%
• Issue #2: how well do individual techniques work?– Evaluated each toolbar as blackbox
– Need to unpack effectiveness of various techniques
• We are developing a toolbar to explore these issues– Developed two new heuristics
– Still needs a name
Our Anti-Phishing Toolbar
• Heuristic #1 – Does it have text input fields?– No text input fields, not phishing
• Heuristic #2 – Content analysis– Based on Robust Hyperlinks by Phelps and Wilensky
– Too many “404 Not Found”
– Create a “lexical signature” for a web page
– Feed lexical signature into search engine to find same page
– Term Frequency / Inverse Document Frequency (TFIDF)• Take the top six terms
Our Anti-Phishing Toolbar
• Heuristic #2 – Content analysis using TF-IDF– Apply TF-IDF algorithm to web page in question
– Feed top six terms into Google
– See if domain of web page in question is in top 30 results• If so, probably not a phish
+
Our Anti-Phishing Toolbar
• Informal results:– 94% accurate
– 6% false positive
– Pretty good, considering it took us 2 weeks to build
• Turns out content analysis works well for anti-phishing– Most scammers modify original web page
– Not enough time for phish page to get high PageRank
• Next steps– Integrate other heuristics
– Evaluate heuristics separately and combined
– Better user interfaces for warning people
Summary
• Usable Privacy and Security increasingly important• Supporting Trust Decisions
– One of our group projects at Carnegie Mellon
– Human-Side of Anti-Phishing• Interviews, Embedded Training, Anti-Phishing Game
– Computer-Side• Email Filter, Testbed, Our Anti-Phishing Toolbar
Questions?
• Alessandro Acquisti• Lorrie Cranor• Sven Dietrich• Julie Downs• Mandy Holbrook• Jason Hong• Norman Sadeh
• NSF IIS-0534406 • ARO D20D19-02-1-0389• Cylab
• Serge Egelman• Ian Fette• P. Kumaraguru (PK)• Yong Rhee• Steve Sheng• Yue Zhang
Usable Privacy and Security Important
• People increasingly asked to make trust decisions– Install this software?
– Trust expired certificate? (“what the !@^% is a certificate?”)
– Share location information?
Everyday Risks Extreme Risks
Hackers, Muggers_________________________________
Identity TheftPersonal safety
Employers_________________________________
Over-monitoringDiscrimination
Reputation
Friends, Family_________________________________
Over-protectionSocial obligationsEmbarrassment
Government__________________________
Civil liberties