1 Cameron Middle School into Cameron College Prep - TN.gov

25
Software Trace and Memory Dump Analysis Presenter: Dmitry Vostokov Memory Dump Analysis Services

Transcript of 1 Cameron Middle School into Cameron College Prep - TN.gov

Page 1: 1 Cameron Middle School into Cameron College Prep - TN.gov

Software Trace and

Memory Dump Analysis

Presenter: Dmitry Vostokov

Memory Dump Analysis Services

Page 2: 1 Cameron Middle School into Cameron College Prep - TN.gov

Prerequisites

Experience in software

troubleshooting and reading

software logs

Advantage: Citrix CDF and

Microsoft ETW trace analysis

including Process Monitor logs

© 2011 Memory Dump Analysis Services

Page 3: 1 Cameron Middle School into Cameron College Prep - TN.gov

Agenda

Memory Dump Analysis Services

Root Cause Analysis Methodology

Software Traces and Memory Dumps

Examples

© 2011 Memory Dump Analysis Services

Page 4: 1 Cameron Middle School into Cameron College Prep - TN.gov

MDA Services

Memory Dump Analysis Audit

Software Trace Analysis Audit (New)

Software Error Reporting Audit

Remote Training

Debugging Bureau

Tool Objects and EasyDbg

Powered by DA+TA

DumpAnalysis.org + TraceAnaysis.org

© 2011 Memory Dump Analysis Services

Page 5: 1 Cameron Middle School into Cameron College Prep - TN.gov

A.C.P. Root Cause Analysis

© 2011 Memory Dump Analysis Services

Artifacts

Checklists

Patterns

Checklists and patterns

as best practices

Iterative and Incremental

Page 6: 1 Cameron Middle School into Cameron College Prep - TN.gov

DA+TA

DA: Dump Artifact / Dump Analysis

Memory snapshots: process, kernel, physical memory dumps

TA: Trace Artifact / Trace Analysis

Software traces: Event Tracing for Windows, logs

© 2011 Memory Dump Analysis Services

Page 7: 1 Cameron Middle School into Cameron College Prep - TN.gov

Spatiality vs. Narrativity

© 2011 Memory Dump Analysis Services

Narrativity

Spartiality

Software

Trace

Memory Dump

Software trace as software narrative,

the story of a computation

Page 8: 1 Cameron Middle School into Cameron College Prep - TN.gov

Tools for Artifact Analysis

Memory dumps:

WinDbg from Debugging Tools for Windows

Notepad (textual debugger logs)

Software traces:

CDFAnalyzer* / CDFControl from Citrix

Process Monitor* from Microsoft

* supports adjoint threads

© 2011 Memory Dump Analysis Services

Page 9: 1 Cameron Middle School into Cameron College Prep - TN.gov

Checklists for Analysis

Memory dumps:

http://www.dumpanalysis.org/blog/index.php/2007/06/

20/crash-dump-analysis-checklist/

Software traces:

http://www.dumpanalysis.org/blog/index.php/2011/03/

10/software-trace-analysis-checklist/

© 2011 Memory Dump Analysis Services

Page 10: 1 Cameron Middle School into Cameron College Prep - TN.gov

Software Behavior Patterns

Memory dump and software trace

Examples: Spiking Thread, Discontinuity

+200 patterns (DA+TA)

DumpAnalysis.org

© 2011 Memory Dump Analysis Services

Page 11: 1 Cameron Middle School into Cameron College Prep - TN.gov

DA: Software Behavior

Memory dump: a memory snapshot

Definition, partial classification and

historical list

Pattern identification case studies

© 2011 Memory Dump Analysis Services

Page 12: 1 Cameron Middle School into Cameron College Prep - TN.gov

TA: Software Behavior

“Imagine you got a software trace from hundreds of modules

you haven’t written or haven’t seen source code of...”

Software trace: a sequence of memory

fragments ordered in time

Definition, and historical list

Pattern identification case studies

© 2011 Memory Dump Analysis Services

Page 13: 1 Cameron Middle School into Cameron College Prep - TN.gov

CDFAnalyzer Filters

© 2011 Memory Dump Analysis Services

Page 14: 1 Cameron Middle School into Cameron College Prep - TN.gov

ThreadsTime

# PID TID Time Message

Time

# PID TID Time Message

© 2011 Memory Dump Analysis Services

Page 15: 1 Cameron Middle School into Cameron College Prep - TN.gov

Adjoint Threads

© 2011 Memory Dump Analysis Services

Time

# PID TID Time Message

Time

# PID TID Time Message (ATID)

Page 16: 1 Cameron Middle School into Cameron College Prep - TN.gov

Significant Event

csrss.exe

winlogon.exe

LogonUI.exe

userinit.exe

Custom events: CDFMarker

© 2011 Memory Dump Analysis Services

Time

# PID TID Time Message

Page 17: 1 Cameron Middle School into Cameron College Prep - TN.gov

Discontinuity

© 2011 Memory Dump Analysis Services

14:23:02.146

14:23:02.345

14:31:10.254

14:31:10.341

Time

# PID TID Time Message

Page 18: 1 Cameron Middle School into Cameron College Prep - TN.gov

No Activity

Expecting messages from Module X

Absence of such messages may

suggest that a process or a thread was

hang / blocked

© 2011 Memory Dump Analysis Services

Page 19: 1 Cameron Middle School into Cameron College Prep - TN.gov

Guest Component

Sudden appearance of an unexpected

module, for example, werfault.exe or

faultrep.dll

© 2011 Memory Dump Analysis Services

Page 20: 1 Cameron Middle School into Cameron College Prep - TN.gov

Statement Current

The flood of messages

Normal case: 15 msg/s

Abnormal case: 3500 msg/s

May point to a CPU spike

© 2011 Memory Dump Analysis Services

Page 22: 1 Cameron Middle School into Cameron College Prep - TN.gov

More Resources

August remote training season:

Accelerated Windows Memory Dump Analysis

Complete Physical Memory Dump Analysis

Visit Memory Dump Analysis Services for registration details:

www.DumpAnalysis.com

© 2011 Memory Dump Analysis Services

Page 23: 1 Cameron Middle School into Cameron College Prep - TN.gov

Free Summer Webinars

The Old New Crash: Cloud Memory Dump

Analysis (June 6th)

Cyber Warfare Memory Dump Analysis

(forthcoming in July-August)

Visit Memory Dump Analysis Services for registration details:

www.DumpAnalysis.com

© 2011 Memory Dump Analysis Services

Page 24: 1 Cameron Middle School into Cameron College Prep - TN.gov

Q&A

Please send your feedback using the contact

form on DumpAnalysis.com

© 2011 Memory Dump Analysis Services

Page 25: 1 Cameron Middle School into Cameron College Prep - TN.gov

Thank you!

© 2011 Memory Dump Analysis Services

Join DA+TA Facebook Group