Continuous Security Testing
-
Upload
ray-lai -
Category
Technology
-
view
179 -
download
1
Transcript of Continuous Security Testing
© 2016 by Ray Lai, Confidential and Proprietary© 2016 by Ray Lai, Confidential and Proprietary
Continuous Security TestingRay LaiDec 7, 2016 (Version 0.2)
© 2016 by Ray Lai, Confidential and Proprietary
SynopsisThis session addresses the technology challenges of continuous security testing to “deliver securely,” and discusses best practices and tooling based on first hand experience in both enterprise and startup environment.
© 2016 by Ray Lai, Confidential and Proprietary
The Problem
How to complete security testing whenever a build is initiated
Develop
Security testDeploy
ImpactUS$10,000 / bug fixedUS$4M data breach (2016) Damage to brand
© 2016 by Ray Lai, Confidential and Proprietary
Security Testing: Now and Then
People• Professional/consultants
Technology• Commercial tools
Process• Central governance• Testing before SW release
Before/Now Desired State
People• Engineering community
Technology• Open source tools• UI and API security testing
Process• Self-serve• Continuous: at build/deploy
time
© 2016 by Ray Lai, Confidential and Proprietary
How
© 2016 by Ray Lai, Confidential and Proprietary
What is “Continuous Security Testing”?
Continuous Security Testing = New Tools?
© 2016 by Ray Lai, Confidential and Proprietary
Continuous Security Testing: Point of View
App Source Codes
Objectives• Efficiency• Feedback loop for developers
Security Metrics
Automated Security Testing
Web App Vulnerability Scanning
Threat Modeling
Secure Build
Build PipelineInput Output
Stakeholders• DevOps• Developers• Quality engineers• Security
• Static code analysis• Vuln scanning• App assessment• Pen test• Risk assessment• Compliance• Operations review• Defect reports• Threat modeling
Challenges: false positives/noise, long build time
© 2016 by Ray Lai, Confidential and Proprietary
Continuous Security Testing: Paradigm Shift
Testing approach• Network vs app security• Front-end vs backend;
integrated test
Tools• Enterprise (integrated), On-
premise
Execution• Testing separated from build
pipeline
Old New
Testing approach• Testing how apps and
systems are connected
Tools• Cloud-based, per pay use• Single-purpose, open source
Execution • DevOps: Test in build phase
© 2016 by Ray Lai, Confidential and Proprietary
How to Deliver Securely?
Exercise 1
You have just completed functional and security testing for your flagship app and platform. However, a security researcher informs you that they can reset user password via SMS, and take over a user’s account.
Options1. Extend test coverage
(resources)2. Enforce pen test before
release (professionals)3. Educate engineers (training)
How would you do differently to deliver securely?
© 2016 by Ray Lai, Confidential and Proprietary
How to Deliver Securely?
Exercise 2
A security researcher reports a Cross-site Scripting (XSS) issue in your portal, with XSS examples in existing static Web pages. Your dev team argues that XSS on static Web pages is not harmful, and any fix will take them at least 1 month.
Options1. Validate if this is a fire drill
(risk management, education)2. Fix any XSS defect
(resources)3. Enforce pen test before
release (professionals)
How would you do differently to deliver securely?
© 2016 by Ray Lai, Confidential and Proprietary
How to Deliver Securely?
Exercise 3
Amazon AWS notifies your security team that your standalone test VM (AMI) has been compromised. While your app team argues that it is an isolated instance, AWS finds traces that it was used to access other AMI instances without permission, and also becomes a DDOS agent.
Options1. Introduce security controls
and security testing for cloud apps (technology, resources)
2. Enforce pen test before release (professionals)
3. Raise awareness (education)
How would you do differently to deliver securely?
© 2016 by Ray Lai, Confidential and Proprietary
How to Deliver Securely?
Recommendation (Past) Continuous Security Testing
Exercise 1 (password reset)
Threat modelingTest for trusted boundaries (connected security)
In build pipeline, spin an app instance to test for end-to-end flow.
Exercise 2 (XSS)
Resolve any XSS defectsRaise awareness
Include UI testing for XSS in automated test suite (build process).
Exercise 3 (compromisedAMI)
Cloud security testing –production pen testCloud security controls
In build pipeline, spin an AMI instance to test AWS security policies and controls.
This is what I would do differently...
© 2016 by Ray Lai, Confidential and Proprietary
Recommendation• Continuous security testing is a journey (various maturity levels)
– Identify your low hanging fruits and priorities– Building community is an art (e.g. contents, outcome and appropriate
engagement)• Be open mind about changes in execution approach
– DevOps -> build pipeline for security– Gradual shift from commercial, integrated tools to open source tools
• Be prepared for the next big wave– Emerging security challenges in IoT and block chain technologies
© 2016 by Ray Lai, Confidential and Proprietary
Appendix: Best Practices
© 2016 by Ray Lai, Confidential and Proprietary
What to Prioritize
https://techcrunch.com/2015/01/22/security-for-startups-in-10-steps/
Prioritize
Security culture
Secure platform
Your Website is front-door
Secure coding
Security controls
Plan for failurePhysical security
Be open with public
Email with multi-factor auth
Threats
© 2016 by Ray Lai, Confidential and Proprietary
Which Tools to Use
brakerman
Static Code Analysis / Vulnerability Scanners
Cloud / Infrastructure Security
Data SecurityConnected Systems
WhatWeb Web Scannerw3afOSSEC Arachni
findBugsFlawFinder
PMD
© 2016 by Ray Lai, Confidential and Proprietary
What to TestMajor Security Vulnerabilities Static Code
AnalysisWeb App Scan Pen Test
OWASP Top 10 Partial Partial PartialHeartBleed-type SSL Issues No No PartialDOS, e.g. due to memory leakage Partial
(Memory leakage)
Partial Partial
DDOS, e.g. bot attack No No PartialCloud hosting security issues No Limited YesNoSQL database security issues No No PartialIoT security issues No Varies VariesBlockchain security issues No No Varies
© 2016 by Ray Lai, Confidential and Proprietary
What to Test (cont’d)OWASP Top 10 Static Code
AnalysisWeb App Scan Pen Test
A1 Injection X X X
A2 Broken Auth and Session Management X X
A3 Cross-site Scripting (XSS) Partial X X
A4 Insecure Direct Object References Partial X
A5 Security Misconfiguration X X
A6 Sensitive Data Exposure Partial X
A7 Missing Function Level Access Control X Partial
A8 Cross-Site Request Forgery (CSRF) X X
A9 Using Components with Known Vulnerabilities
Partial
A10 Unvalidated Redirects and Forwards Partial X
© 2016 by Ray Lai, Confidential and Proprietary
How to Execute (Build Pipeline for Security)Web App
ScanDeploy to Production
Security Assessment
Deploy to Production
Build Pipeline
Penetration Test
Runtime Container
Security Test Client
Web App Scan
Before
New
Separate, serial events
Continuous integration process to create temporary containers for security testing
© 2016 by Ray Lai, Confidential and Proprietary
Security Build Pipeline (Example)
App Container
Jenkins Server
App Container
Docker Swarm
App Container
Security Scanner (Docker)
Shared Data Container / Data Volume
Apps
Data
Pre-requisites• Apps deployed in docker containers• App data (synthetic or scrubbed
production data) in shared data containers
• Parallelize testing tasks in Jenkins
Jenkins Setup• Create docker swarm (apps and data)• Launch docker swarm• Create security test client (security
scanner)• Launch security test client• Post test results in shared data
container or data volume• Shut down all docker containers
Gauntlt, ClamAV, Brakerman, OWASP ZAP
© 2016 by Ray Lai, Confidential and Proprietary
What to Look For: Web App Security Testing
Session Management• HTTPOnly and SECURE
flags (cookies)
UI and App• XSS, CSRF
Data• Injection vulnerabilities
Traditional Approach What’s New
API• API token robustness;
playback and Man-in-the-middle attacks
Data theft• Data privacy issues (in transit
or at rest)
NoSQL database• Access control issues
© 2016 by Ray Lai, Confidential and Proprietary
What to Look For: Platform and Cloud
Platform Infrastructure• Network security, e.g. port
scan, mis-configuration• Default passwords and
settings
Identity management• SSO; access control
Compliance• Security certification, e.g. PCI
Traditional Approach What’s New
Platform Stack• Timely patches• Core security, e.g.
Heartbleed bug• DDOS
Data security• Storage security, e.g. S3
Compliance• Public cloud (e.g. AWS)
security policies and controls
© 2016 by Ray Lai, Confidential and Proprietary
How to Measure Success
Defect Density Threat Model Custom Risk
Scores
OWASP Top 10
WhiteHat WSI Security Index
Cenzic HARM Scores
HP Fortify Scores
Internal Metrics
Financial management app has a defect density of 15.23 (5M LOC)
25 attack vectors Biz risks: 5.2Tech risks: 7.9
Tech risks reflect # of total vulnerabilities from scanning/testing
Industry
Example:
For benchmarking
© 2016 by Ray Lai, Confidential and Proprietary
How Do I Get Trained
Traditional Channels What’s New
Security Certification• CEH, CISSP, etc.
Conferences• RSA, OWASP AppSec,
BlackHat, DEFCON, etc.
Product training• E.g. Fortify
Online Resources for the Impatient
• Youtube• Udemy, Coursera, etc.
Community• Meetups or user groups, e.g.
OWASP