Post on 02-Jan-2016
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 1
Aegis Research Corporation
Not for Public Release
Intrusion Tolerance UsingMasking, Redundancy and Dispersion
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001
Janet Lepanto
William Weinstein
The Charles Stark Draper Laboratory, Inc.
Aegis Research Corporation®
Aegis Research CorporationAegis Research Corporation
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 2
Aegis Research Corporation
Not for Public Release
Overview
• Objectives and Assumptions
• System Design
– Overview
– Server Configuration Policy
– Configuration Monitoring
– Transaction Mediator
• Testing• Demonstration• Status and Near-Term Plans
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 3
Aegis Research Corporation
Not for Public Release
Objectives and Assumptions
• Objectives
– Apply fault-tolerant design concepts to support intrusion tolerance
– Minimize loss of data confidentiality and integrity in the presence of a successful attack on one of the servers
– Tolerate attacks whose specific signatures are not known a priori
– Employ only a small set of trusted components to protect a large set of untrusted unmodified COTS servers and databases
– Employ redundancy for both intrusion tolerance and performance
• Assumptions
– Attacker desires stealth so transaction rates will be relatively low
– Attacks employing high transaction rates and recognizable signatures will be addressed by front-end firewalls and/or other intrusion detection mechanisms
– Exploitation of latent vulnerabilities will require more than a single transaction
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 4
Aegis Research Corporation
Not for Public Release
System Design Overview
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 5
Aegis Research Corporation
Not for Public Release
Technical Approach
• Mask fingerprints of gateway and origin servers so that an attacker cannot probe over network to determine
– OS of gateway, or origin servers
– Implementation of any origin server
• Distribute each client’s transactions among origin servers such that the client cannot predict which server will handle a transaction
• Periodically inspect each origin server for configuration anomalies that might indicate that attack transactions have occurred
– Reconfigure server to “clean” state if anomalies are detected
• Log transactions to back-end database so that data written by a compromised origin server can be reconstructed
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 6
Aegis Research Corporation
Not for Public Release
Basic Architecture-Phase 2E
xter
nal
WA
N
ExternalFirewall
DataBase
TransactionMediator
Gateway
Sw
itch
ed I
P
Server(1)
Server(N)
Server(2)
Configuration Manager
Sw
itch
ed I
P
COTS
Trusted
Other
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 7
Aegis Research Corporation
Not for Public Release
Configuration Management
Config Manager Origin Server
CM AgentConfiguration Management
Routines
Daemon, tailored to origin server OSMonitors server config info as directed by CM
Executes algorithms on origin server,(directory searches, hash calculations, etc.)
Reports status and transfers requested data to CM
Manages operation of CM AgentControls local IP switchFunctions as a client only
Manages Gateway operationDetermines servers in active set
Controls server utilization by updating “use/don’t use” status in Gateway
Origin Server Configuration
Templates
Structure of file system directory hierarchyExistence of specific files in specific directoriesHash integrity check of specific filesConfiguration data is dependent on server OS
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 8
Aegis Research Corporation
Not for Public Release
Origin Server Configuration Policy
• Explicitly bind the http daemon and the CM agent to the external interface
– Don’t allow direct access from localhost to either one of the ports
– This prevents an attacker from hijacking the server port without killing the existing server daemon
• Run HTTP daemon as “nobody” (or the equivalent)
– Don’t run daemon as root
– Don’t allow “nobody” shell access
– This prevents an attacker from gaining immediate root privileges
• Run CM Agent daemon as root
– It needs most root privileges anyway, and giving it an account name would potentially make it easier to find
• Configure server-side scripts (cgi-bins) as read/execute only
• Don’t allow execution from the temp directory
• Don’t allow user-level processes to access RAM directly
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 9
Aegis Research Corporation
Not for Public Release
Server -Side Scripting Policy
• All operational data is maintained on the back-end database server– All origin servers must remain functionally equivalent
– No client transaction data can persist on an origin server outside the duration of the transaction
– Because server-side scripts are constrained this way, they have no need to invoke root-level functions
• Server-side scripts– Will move data to and from the back-end servers via Transaction Monitor proxies
• File operations
• ODBC commands
– Access and user authentication• Via web server
• Via native database access management
• Back-end server administration– Done directly on servers
– Mediator can be configured to block administrative database operations
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 10
Aegis Research Corporation
Not for Public Release
CM Agent – Data Flows
Input Buffer
MUTEX
CM Agent on Origin ServerCommands & reference data from CM
Configuration check results
Monitor & Control Flags and Reference Data
Tier 1 Monitor Thread
Tier 2a Monitor Thread
Tier 2b Monitor Thread
Tier 3 Monitor Thread
Shared Memory
Shared Memory
Shared Memory
Shared Memory
Shared Buffer
MUTEX
Shared Memory
Executive Thread
Config Man
Other Exec msgs to CM
Other info between Exec and Monitors
Output Buffer
MUTEXCM Interface
Thread
Heartbeat msg
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 11
Aegis Research Corporation
Not for Public Release
Tier 1 Monitor
• Checks– Selected processes for proper name, status and PID value
• Use process config file to specify which process must be present (normal http daemon, agent daemon, and maybe others) and which other processes are allowed
• Use direct invocation of system calls for process ID, status checks and running privileges (this also validates that the server is running as “nobody”)
• Response to check fail– Specified in tier 1 config file (per each process) — responses include
• Leave server operating, invoke tier 2b check directly
• Take server off-line, invoke tier 3 check
• Rationale– Attempts to subvert the web server daemon or the monitoring agent processes will be
detected rapidly, before additional transactions occur that could allow further exploitation
Origin server activePeriod ≈ 1 second
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 12
Aegis Research Corporation
Not for Public Release
Tier 2a Monitor
• Checks– Certain portions of the directory structure
• Server-side script files, html server files, certain system files
• All necessary files present, no extraneous files present
• Verify mod times on files
• Verify server configuration parameters
– Server-side scripts are read/executable only
– Temp directory scripts are read/write only
• Response to failure check– Take off-line and check logs to see if config changes were made by sys admin
• Rationale– Monitors for most likely points of exploitation and also picks up inadvertent
configuration changes by a system administrator
Origin server activePeriod ≈ 10 seconds
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 13
Aegis Research Corporation
Not for Public Release
Tier 2b Monitor
• Checks– Hash of core files from Tier 2a
– Verify server configuration parameters
• Binding of http parent daemon to external interface
• User “nobody” has no shell access
• Response to failure check– Take off-line and do full tier 3 checks
• Rationale– Fairly frequent and thorough check of critical files for subtle changes
Origin server activePeriod ≈ 5-10 minutes
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 14
Aegis Research Corporation
Not for Public Release
Tier 3 Monitor
• Checks– Full forensics, memory and file checkout
• Hash check all files
• Dump and check memory
• Response to failure check– Determine whether database might need recovery
– Scrub the platform
• Rationale– Detect whether a sophisticated attack has occurred and what damage
might have been done
Origin server inactivePeriod ≈ 1 hour or more
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 15
Aegis Research Corporation
Not for Public Release
Back-end Operations
Back-end Databases
Origin Server
SQL Database(Brand Y)Web Server
Static Web Pages
Web Content
COTS
OASIS Developed
User Developed
Comms interface to Origin Servers & Config Man
Configuation Manager
Transaction Mediator
SQL Database(Brand X)
Back-end File Servers
File SystemBack-end
File ServerInterface
Scripts
ODBC Manager Interface
Proxy
Comms interface to Trans Monitor
App Server
InterfaceProxy
File Server
InterfaceProxy
Back-end App Servers
Applications
ODBCDriver X
ODBCDriver Y
ODBCManager
Monitor Proxy
TransactionLog
Database
Back-end App Server
Interface
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 16
Aegis Research Corporation
Not for Public Release
Proxying ODBC
Back-end Databases
Origin Server
SQL DatabaseServer
(Brand X)
Transaction Log Database
ODBCDriver forBrand X
SQL DatabaseServer
(Brand Y)
ODBCManager
Comms interface to Trans Monitor
Web Server
ScriptsStatic Web Pages
Web Content
COTS
OASIS Developed
User Developed
CGI scripts produce SQL command strings to access back-end database—SQL flavor used depends on DB product accessed (same as if Origin Server accessed DB directly)
Supports traceability of ODBC commands to Origin servers and back-end database rollback
ODBC Manager interface proxy provides normal ODBC manager interface to CGI scripts
Comms interface to Origin Servers & Config Man
ODBC Manager Interface Proxy
Monitor Proxy
Config ManagerProvides back half of ODBC manager proxy—Manages operation of mediator
Transaction Mediator
ODBCDriver forBrand Y
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 17
Aegis Research Corporation
Not for Public Release
Multi-Tier Database Connectivity Standards
• Standard SQL: International Standards Organization (ISO) and National Committee for Information Technology Standards (NCTIS) recognize a standard dialect of SQL known as SQL-92, published under similar and parallel standards ISO/IEC 9075-1992 and ANSI/NCTIS X3.135, respectively. Sub-levels of compliance exist within SQL-92 (Entry, Standard, Advanced).
• Call Level Interface: Call Level Interface (CLI) dates from 1995, and provides a standard set of API bindings for compiling languages (C, COBOL, Fortran, etc.) to guarantee portable execution of many SQL commands. A product of the Open Group, CLI is now an ISO standard extension of SQL (ISO/IEC 9075.3-1995).
• Microsoft’s Open Database Connectivity (ODBC) extends most of the features defined under CLI.
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 18
Aegis Research Corporation
Not for Public Release
System Testing
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 19
Aegis Research Corporation
Not for Public Release
Verification Testing Approach
• Subject the system to known scanning tools to determine if the mechanisms to thwart those scans are implemented properly
• Subject the system to known attacks to evaluate how it reacts to various types of attacks (e. g., measuring the relative time to success for an attack directly on server X vs. the same attack on server X operating in our OASIS architecture).
• Subject the system to modifications of known attacks developed to exploit knowledge of the architecture and operation of the system
• Setup system as a honeypot to observe character of real attacks and system response to those attacks
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 20
Aegis Research Corporation
Not for Public Release
Verification Testing Status
• TCP/IP masking verification (Gateway and Origin Servers)– Fingerprinting attempts unsuccessful (nmap, queso, p0f, siphon)
– Fingerprinting attempts using ICMP errors unsuccessful (ICMP Port Unreachables)
• Proxy Scrubbing Verification– HTTP header cleanup successful
– Exception: Basic parameters of the HTTP request were cleaned, except for the HTTP Version (HTTP/1.0)
• Web page error documents– Intentional requests for non existent documents (404 Errors) allows
enumeration of origin servers
• Common Hacks Unsuccessful– Whisker (Times out due to the transaction dispersion algorithm)
– Win NT Unicode Exploit (Due to a patched server)
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 21
Aegis Research Corporation
Not for Public Release
Verification Testing Plans
• Explore other ICMP techniques to discover the OS• Explore Reaction to Common Hacks
– Whisker (We are modifying to ignore time settings and server identification responses)
– WIN NT ISAPI Overflows• Index Server/Unicode Overflow Variant (Currently developed to stop
http process, will be modified to gain privileges)
• Print Server (Can overflow stack, being modified to initiate outbound trojan connections)
• Explore attacks against content– Content will be added to allow for a real world scenario test
– CGI scripts will be evaluated as a means of gaining access
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 22
Aegis Research Corporation
Not for Public Release
Demonstration
• Transaction Dispersion• HTTP Whitening
Inte
rnet
GatewayOASIS Santa
Fe
Sw
itch
ed I
P
Solaris 2.7Apache 1.3.19
Windows 2000Apache 1.3.19
Open BSD 2.8Apache 1.3.19
Macintosh OS 9.1Personal Web Server
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 23
Aegis Research Corporation
Not for Public Release
Program Status and Near-Term Plans
• Gateway w/redundant servers operating– Accessible over the Internet
– Independent sniffer monitoring traffic between Internet and Gateway
– Aegis conducting attack testing
– Plan to make system available as a target (to other OASIS projects)
• Server agents– Common structure is designed
– Coding underway for Unix and NT versions
• Transaction Monitor– Basic structure designed
– ODBC proxy mechanism being implemented
DARPA OASIS PI Meeting – Santa Fe – July 24–27, 2001 Slide 24
Aegis Research Corporation
Not for Public Release
Task Schedule
1.1 Dev & Implmt Fingerprint Masking
1.2 Dev & Implmt Dynamic Assign
1.3 Dev & Implmt Config Evalution
1.4 Integration & Experimentation
1.5 Program Management
2.1 Dev & Implmt Transaction Mediator
2.2 Dev & Implmt Assess & Rollback
2.3 Integration & Experimentation
2.4/3.6 Dev Sync of Redundant Databases
2.5 Program Management
3.1 Refine Fingerprint & Dynam Assign
3.2 Implement Sync & Voting
3.3 Refine Config Assessment
3.4 Integration & Experimentation
3.5 Program Management
4.1 Refine Sync & Voting
4.2 Implmt Distribution of Servs & DBs
4.3 Integration & Experimentation
4.5 Program Management
Task Name
7 10 1 4 7 10 1 4 7 10 1 4 7 10 12
CY 2000 CY 2001 CY 2002 CY 2003
Phase 1 – Basic Architecture
Phase 2 – Extended Architecture