UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the...
Transcript of UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the...
![Page 1: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/1.jpg)
UI and Security: Two Sides of the
Story
Wenke Lee
Georgia Institute of Technology
![Page 2: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/2.jpg)
Who Cares About UI?
•Users do!
–That is how they use computers
• “Users are the weakest link in security”
•So, let’s look at the interplay between UI
and security
1
![Page 3: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/3.jpg)
UI Can Both Hurt and Help Security
•New UI features can introduce new
vulnerabilities
–Not just in the UI code but also in the
underlying system
•UI can be leveraged to improve security
–Infer user intent and apply the
appropriate security policy
2
![Page 4: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/4.jpg)
The Insecurity Because of UI
•Accessibility introduces new paths of
privileged access to system resources
•Powerful window manipulation permission
with accessibility on Android can hide
attacks from users
3
![Page 5: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/5.jpg)
Computer Accessibility (a11y)
• For the person with disabilities
–Visually impaired•Text-to-Speech reader
–Hearing impaired•Captioning service
–Motor impaired•Voice Commander
–Keyboard impaired•On-screen keyboard
4
![Page 6: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/6.jpg)
Accessibility Library
• OS opens API for developing A11y features
–Available in Windows, OS X, Ubuntu, iOS, Android,
etc.
• Capabilities
–Read UI states of the system
–Perform actions on UI elements
•Click
•settext ()
•etc.
5
![Page 7: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/7.jpg)
Security Implications of A11y
• Creates new I/O Paths
• Break basic/traditional assumptions on I/O
–Input comes from the user
•Through a11y interface, a program can send input
event to the application
–Output can only be seen by the user
•A11y interface allows to a program can read
output of the other applications
6
![Page 8: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/8.jpg)
Traditional I/O Paths in OS
7
App
App
Ouptut
Input
Handler
Regular
Input DevicesScreen
Output
OS
![Page 9: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/9.jpg)
A11y Added New I/O Paths to OS
8
Launch
Google.com!
1. User Speaks
2. Voice commander translate it into
machine command
3. OS delivers command to the app
(a11y library)
4. App is controlled by Voice
1) Click address bar
2) Type google.com
![Page 10: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/10.jpg)
A11y Added New I/O Paths to OS
9
Process
Output
Assistive Technology
OS
App
Process
InputApp
Ouptut
Input
Handler
A11y
Library
Alt. input through a11y
Regular
Input DevicesScreen
Output
Original I/O path
A11y Input
(Voice)
![Page 11: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/11.jpg)
A11y Added New I/O Paths to OS
10
1. App shows text output 2. Screen Reader gets app output
3. OS gets audio playing request4. User receives audible output
Prepare audio for text…
What’s for
Lunch?
![Page 12: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/12.jpg)
A11y Added New I/O Paths to OS
11
Process
Output
Assistive Technology
OS
App
Process
InputApp
Ouptut
Input
Handler
A11y
Library
Alt. output through a11y
Regular
Input DevicesScreen
Output
Original I/O pathA11y Output
(Speaker)
![Page 13: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/13.jpg)
Required Security Checks
12
• Security checks must be put in the right places
–Does a11y input really comes from the user?
• Checks can be placed in three different level
–Assistive Technology (processor of alternative I/O)
–Operating System
–Application (protect themselves from alternative I/O)
![Page 14: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/14.jpg)
At Assistive Technology (AT) Level
13
Launch
Google.com!
Required checks at AT levelIs the voice from real human?
If not, machine can access it!
Is the voice matched with registered
user?If not, any other human user can access it!
![Page 15: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/15.jpg)
At OS Level
14
Required checks at OS levelIs this assistive technology allowed to
access a11y?If not, any program (possibly malware) can
access it!
![Page 16: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/16.jpg)
At Application Level
15
Required checks at application levelShould I react to input from a11y
features?
In particular for security sensitive UI
actions!
![Page 17: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/17.jpg)
Evaluating A11y Security in OSes
• Objective
–Check if OSs are secure under attacks through new
I/O paths created by supporting A11y
• Method
–Analyze OS for accessibility features
•Programmatic access to I/O event
•Voice commander, password viewer, etc.
–Test existence of required security checks
•If not, try to launch an attack
16
![Page 18: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/18.jpg)
Evaluating A11y Security in OSes
• Target–4 Major OSes
•MS Windows 8.1, Ubuntu 14.04 Linux
•iOS 6, and Android 4.4
• Focus–Try to evaluate OS default settings
•AT-level check
–Voice Commander
•OS-level check
–Programmatically controllable I/O
•App-level check
–We do not perform the evaluation …
17
![Page 19: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/19.jpg)
Evaluation on A11y Input
18
PlatformAT-level check
(voice commander)
OS-level
Security CheckVulnerable?
WindowsNone
(Speech Recognition)UIPI YES
Ubuntu N/A None YES
iOS 6None
(Siri)None YES
AndroidVoice Authentication
(Moto X)
User Settings
RequiredYES
![Page 20: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/20.jpg)
Evaluation on A11y Output
19
PlatformReading of
UI Structure
A11y leaks on
screenshot
Password
protectionVulnerable?
Windows UIPI Yes Yes YES
Ubuntu None NoYes, but
incompleteYES
iOS 6 N/A Yes N/A YES
AndroidUser Settings
RequiredNo
User Settings
RequiredYES
![Page 21: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/21.jpg)
Attacks for missed checkpoint
• We tried to launch attacks if any of security check is missing
• We found 12 new attacks
–Windows (3)
•2 Privilege escalation, 1 password leak
–Linux (2)
•Bypassing process boundary, password leak
–iOS (4)
•Bypassing sandbox and authentication
•Privilege escalation, Password leak
–Android (3)
•Bypassing sandbox and authentication
•password leak
20
![Page 22: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/22.jpg)
Attacks on Voice Commander
• Voice commander accepts non-human voice
–Any app capable to play audio can send command
•Broken assumption: input comes from the user
–No authentication
•Windows Speech Recognition
•Siri
•Google Now
–Voice authentication in presence
•Moto X
–Vulnerable to replay attack
21
![Page 23: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/23.jpg)
Privilege Escalation in Windows
• Malware runs as normal user can execute
Speech Recognition
• Speech Recognition automatically launches with
administrative privilege
–Let A11y user control admin stuff …
• Malware can get admin privilege by sending
voice command to Speech Recognition
22
![Page 24: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/24.jpg)
Take Control Over Other Apps
• A11y library allows a program send input to the other
apps
–Broken assumption: input comes from the user
• Privilege escalation
–Windows
•Send click to a security-sensitive dialog
• Bypassing app sandbox
–iOS and Android
•Sending programmatic input to the target app
23
![Page 25: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/25.jpg)
Stealing Password!
• Applying image processing on screenshot leaks
password string.
24
A) Before clicking
Eye
B) After clicking Eye
![Page 26: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/26.jpg)
Discussions
• Root-cause
–Maximizing compatibility
•The UI is expected to run as if it gets the real input
on a11y request
•Programmatic input processed as same as the
real one
25
Real Touch Click A11y Click
Intermediate func onTouchEvent() performA11yActionInternal()
Final handler in UI performClick() performClick()
![Page 27: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/27.jpg)
The same PerformClick() is called
26
![Page 28: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/28.jpg)
Discussions
• Root-cause
–Problems when it handled user input and ATK input
differently
•On gksudo dialog, copytext() works while Ctrl-C
does not work!
•New implementation could miss security checks.
27
GTK::CopyText() {
if(text->isVisible)
return text
else
return null;
}
ATK::CopyText() {
…
return text
…
}
![Page 29: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/29.jpg)
Discussions
• Root-cause–No correct authentication for alternative input
• E.g., any program can send fake voice …
–Technical & economical difficulty
•Possible solution for voice authentication
–Liveness check
–Challenge-response
•Practical issues
–Processing power
–Power consumption
–Etc.
28
![Page 30: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/30.jpg)
Discussions
• Root-cause
–Weak access control on a11y libraries
•Windows: None
•OS X : None
•Ubuntu: None
•iOS 6 : None -> patched in iOS 7
•Android: User settings
–Not enough …
29
![Page 31: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/31.jpg)
Leveraging UI to Improve
Data Protection?
![Page 32: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/32.jpg)
What data is important/valuable?
Ask the user?
What? Aren’t They the Weakest Link?
![Page 33: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/33.jpg)
What data is important/valuable?
User-Intent Monitoring of (Text-Based) Networked Applications
![Page 34: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/34.jpg)
User Intent from UI
• User interacts with computer using
input/output hardware• Input: Keyboard, Mouse
• Output: Display screen
– Feedback loop in user interaction
33
![Page 35: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/35.jpg)
Capturing User Intent
• Observation
–User verifies what their input by on-screen display
• A New Security Policy
–What You See Is What You Send (WYSIWYS)
•On-screen text is user-intended
•Only allows traffic that matches on-screen text
Hi, there!
Hey, send me $100
34
![Page 36: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/36.jpg)
What You See Is What You Send
• WYSIWYS (Facebook example)
35
![Page 37: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/37.jpg)
What You See Is What You Send
• Supposed we have a security monitor for WYSIWYS
–Compares outgoing traffic data with data on
application GUI
–Needs to query application for data on its GUI
–If the application is already compromised
•It can lie about data on GUI
• Cannot trust application GUI
• The security monitor must have control over “GUI”
36
![Page 38: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/38.jpg)
Security Overlay
Combined Screen
On-screen text is always same with captured
text on the security monitor.
37
![Page 39: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/39.jpg)
Security Overlay
• Only re-draws certain elements, e.g., editbox
–Exactly same location, size, and color
–Can support rich-text
•Font, size, color, style, etc.
• Passive UI
–It does not gets any user input
–Content updated after each applications gets input
–Support selection, copy/paste, spell correction, auto-
completion, etc…
38
![Page 40: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/40.jpg)
UI Monitor
• Uses library for UI Testing (UIAutomation)
39
![Page 41: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/41.jpg)
System Architecture and TCB
40
![Page 42: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/42.jpg)
Threat Model
• Hypervisor and security VM is fully trusted
–Assumes VM escape is impossible
• Hardware input devices are trusted, and attacker has no
physical access
–Attacker cannot forge hardware input event
41
![Page 43: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/43.jpg)
Threat Model (Cont’d)
• All hardware input event is interposed at hypervisor first,
then delivered to User VM
–Security VM cannot miss hardware event, and User
VM cannot emulate it
• Completely distrust User VM
–Allows all attacks including Kernel-level malware
•UI monitor is untrusted
42
![Page 44: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/44.jpg)
Capturing User Intent
• Extract all required text from Secure Overlay when
traffic-triggering event happens
–Store it to Authorization DB for enforcement at
network level.
43
![Page 45: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/45.jpg)
Application-Specific Logics
• User Intent Signature
44
![Page 46: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/46.jpg)
Application Examples
45
![Page 47: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/47.jpg)
Paypal Example
• Paypal “Send Money”
46
What user sees when sends money Outgoing network traffic
![Page 48: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/48.jpg)
Paypal Example
• Capturing User Intent
47
When Continue is clicked,
Stores e-mail and amount
{
“ACTION” : “Paypal Send”,
“Recipient” :
“Amount” : “1.00”
}
Authorization Vector
Store it to
Authorization DB
![Page 49: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/49.jpg)
Paypal Example
• Monitoring Network Traffic
–Apply Deep Packet Inspection
48
{
“ACTION” : “Paypal Send”,
“Recipient” :
“Amount” : “1.00”
}
Authorization VectorPasses the traffic only if it is matched
with previously authorized data in DB
![Page 50: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/50.jpg)
Security Overlay For Messaging App
49
![Page 51: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/51.jpg)
Summary
• Security is about data protection
–What data is important to user?
–Usability is the key
•Do users have to change workflow?
• Security overlay
–A systems mechanism
–Monitors user intents through on-screen UI data, and security policy
•Integrity and confidentiality protection
–Transparent
–General
•Applicable to class(es) of applications
50
![Page 52: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/52.jpg)
Biometric Authentication
Image Credit: IBM Future of Identity Study
![Page 53: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/53.jpg)
Challenges
• Privacy Issues• No existing method for
remote biometric-based authentication on encrypted data
• Current practice is storing raw biometrics in the authentication server
• Recent database breaches make people reluctant to give their biometrics to most services
• Stolen Biometrics (Live?)
• You cannot replace a biometric
• Most popular biometric sources (e.g. face, voice) are easily stolen
• Systems like MasterCard’s Smile-to-pay or AliPay’s Blink-to-pay are open to large-scale replay attacksthrough stolen biometrics
![Page 54: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/54.jpg)
• Allow biometric matching
over encrypted biometric
data on remote
authentication server
which enables recovery
Protecting Privacy
![Page 55: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/55.jpg)
• Live-biometrics
– Enrollment & recovery
– Ask users to read out random and
funny English phrases in
CAPTCHA format (rtCapctcha)
• Answer question being asked
• Random to prevent replay
attacks
Liveness Detection
![Page 56: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/56.jpg)
ISTC-Adversarial Resistant Security Analytics
![Page 57: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/57.jpg)
Web Interface
User
AVPass
Research
Module
SHIELD
LSTM-PT
model
instancemodel
instancemodel
instance
model
instancemodel
instancemodel
instance
model
instancemodel
instancemodel
instance
dockermicroservice
Storage
…
MLsploit
![Page 58: UI and Security: Two Sides of the Story - CSIRO Research€¦ · UI and Security: Two Sides of the Story Wenke Lee Georgia Institute of Technology. ... Voice commander translate it](https://reader036.fdocuments.us/reader036/viewer/2022062506/5f03d1937e708231d40aec32/html5/thumbnails/58.jpg)
Credits
• Simon Chung
• Brendan Dolan-Gavitt (NYU Poly)
• Yeongjin Jang (Oregon State)
• Bryan Payne (Netflix)
• Chengyu Song (UC Riverside)
• Erkam Uzun
• Tielei Wang (now a start-up in China)
• Carter Yagemann
57