DARPAJul 2001

1

A Binary Agent Technology for

COTS Software Integrity

Anant AgarwalRichard Schooler

InCert Software

DARPAJul 2001

2

Agenda Objectives & Approach Prototype Progress:

Robustness Fine-Grain Application Monitoring Solaris Investigation

Integration Opportunities

DARPAJul 2001

3

Objectives & Approach Focus on:

Deployed applications - not just for development, QA phases.

Inside the application - not just externally-visible behavior.

Approach: Run-time execution monitoring. Binary instrumentation to inject probes into

release-built executables.

DARPAJul 2001

4

Targets & Assumptions Similarity between explicit attacks and

accidental faults. Assume system-level mechanisms in-

place - not guarding against replacement of entire executable, compromise of OS, etc.

DARPAJul 2001

5

Major Tasks:

Three Major Components in the Prototype: Core technology for customizable agent

insertion into Windows NT. (And now Solaris.)

Anomaly detection and reporting. Rapid recovery and problem pinpointing.

DARPAJul 2001

6

AGENTtest al,0x3jnz 0x1143

AGENTadd ebx,ecxjc 0x1101

AGENTshr edx,0x1add ebx,edx

AGENTtest al,0x3jnz 0x1143

AGENTinc eaxadd ecx,ediadd edx,esicmp eax,0xa

1

2

3 4

5

... 1 2 4 5

while ((c = ++ci)) { INSTRUCTION_ITERATOR ii = c->Instructions(); while ((inst = ++ii)) inst->Lift(null_state); while ((inst = ++ii)) inst->Lift(null_state);

while ((c = ++ci)) { INSTRUCTION_ITERATOR ii = c->Instructions(); while ((inst = ++ii)) inst->Lift(null_state); while ((inst = ++ii)) inst->Lift(null_state);

Binary Instrumentation At each code block, record progress of program execution. Snap program/system state based on policy/action.

DARPAJul 2001

7

Major Components

Snapshot Files

Snapshot Files

Trace Reconstruction

Trace Reconstruction

•Block sequence

•User logging

•Post-Mortem info

Map FilesMap Files

InstrumentationEngine

InstrumentationEngine

ExecutablesExecutables Instrumented

Executables

InstrumentedExecutables

•Block->Address Map

Debug Info

Debug Info

•Address<->Line Map

•Source Module Name

Trace(XML)

Trace(XML)

•Source Line/Module

•Thread

•Annotations

Platform-dependent

inte

rface

inte

rface

Service

Runtime

DARPAJul 2001

8

User Interface

DARPAJul 2001

9

Robustness Our runtime failures should not bring down

the user’s application! We should be robust in the face of both our

own bugs, and external problems, like running out of memory or disk space.

DARPAJul 2001

10

Robustness, cont. Some techniques:

Limit usage of and interference with user-level facilities, like malloc, higher-level file IO, even stack allocation.

Exception handling. (But watch out for nested exceptions from exception-handling context…)

Desperation buffers. Lock ordering to avoid deadlocking with user

code.

DARPAJul 2001

11

Runtime Architecture

Instrumented

Application

Instrumented

ApplicationServiceServiceSnap requests *

Register with service

Read initial options

Event notification *InCert runtim

ebuffers

•Trace

•Memory Dump

•Environmental Info (XML)

Snapshot file

Snapshot file

User Extension DLL •SNMP e-

mail

•SMTP trap

•HTTP

•FTP

•etc.

Attack!

DARPAJul 2001

12

Service Functionality File Management - compression Notification & transportation Heartbeat: is instrumented application still

alive? (Auto-notify, and even auto-kill!) Investigating:

Monitor distributed applications Monitor un-instrumented components.

DARPAJul 2001

13

Fine-Grain Application Monitor

M o n it o re dA p p l ic a t io n

R u n t i m e

S e r v ic e

M a c h in e 1M a c h in e 2

M a c h in e 3

M o n i to r ( O p e r a t o rC o n s o le )

M o n i to r ( A d m in

C o n s o le )

DARPAJul 2001

14

Solaris Investigation New binary platform: SPARC ISA (delay slots, register

windows), COFF format, ELF/STAB debug format, Solaris signal interface, TSD, etc.

Compilers: Forte (SunPro) & gcc. Some new issues:

64 bit support. How to hook runtime (LD_PRELOAD). How to get relocation info (no /fixed:no). Interposition (vs. Detours).

Balance between using Solaris-specific features, and staying generic-Unix-portable.

DARPAJul 2001

15

Integration Opportunities Implementation mechanism for higher-level policies:

Currently hard-wired to trace & snap. Would like user to be able to specify other policies,

triggers and actions: pattern-action language, security automata, etc.

Integration with full-system monitoring: OS-level (e.g kernel, system call hooking) Middleware, Scripting languages Network, Database