Windows Kernel Debugging

Sisimon Soman

Agenda

• Introduction to BSOD• Common approaches to analyze Windows

crash.• Story 1 – Memory corruption.• Story 2 – Stack Overflow• Story 3 – Deadlock • Story 4 - Memory Leak• Q&A – Bring your dumps to discuss here.

Why BSOD

• Cause by buggy drivers.• Drivers share address space with Kernel.• Alternate stack to write dumps.• KeBugcheckEx() – Turn off interrupts.– SMB box say all processor I am dying..– Paint blue screen and write dump to pagefile.

• Winlogon role in writing crash dump.• Type of crash dumps.

Common approaches to Kernel issue

• What bugcheck code says• Which instruction cause crash• How the stack looks at the time of crash• Is third party drivers involved in stack• The IRP which cause the crash.• Is crash or system hung.

Story 1– Memory Corruption

• Driver Verifier– Heap Corruption– Special pool check (Guard Page)

• WinDBG commands to demo.– !analyze –v– kv– u– !pool

Pool corruption continued..Make sure the pool chaining is correct

kd> !pool bc00248c– Pool page bc00248c region is Paged session pool– bc002000 size: 90 previous size: 0 (Allocated) Gla@– bc002090 size: 10 previous size: 90 (Allocated) Glnk– bc0020a0 size: 20 previous size: 10 (Allocated) Vtfd– bc0020c0 size: 8 previous size: 20 (Free) Gtmp– bc0020c8 size: 38 previous size: 8 (Free ) Usqm– bc002100 size: 28 previous size: 38 (Allocated) Gldv– bc002128 size: 58 previous size: 28 (Allocated) GFil– bc002180 size: 198 previous size: 58 (Allocated) Uspi Process: 856b2a58– bc002318 size: 18 previous size: 198 (Allocated) Uspi Process: 856b4528– bc002330 size: 90 previous size: 18 (Allocated) Gla@– bc0023c0 size: c0 previous size: 90 (Allocated) Gla4– *bc002480 size: 80 previous size: c0 (Allocated) *Usms P

Stack Frames in Windows• The purpose of EBP register – Frame Pointer.• Function parameter, local variables in stack

and, it’s the same thread stack (one stack per thread).

• EBP + offset points to function parameter• EBP – offset points to local variables.• X64 Stack frames.

Stack Frames continued..• 0:000> u Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1• Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1 [d:\sisimon\working area\

presentations\windbg\simple app crash3\simple app crash3\simple app crash3dlg.cpp @ 154]:• 00401580 55 push ebp - First Push EBP to stack• 00401581 8bec mov ebp,esp Set new ebp as current esp• 00401583 83ec14 sub esp,14h make enough room for local variable• 00401586 894dec mov dword ptr [ebp-14h],ecx <- Copy the this pointer to stack first

element for the current stack frame• 00401589 c745f400000000 mov dword ptr [ebp-0Ch],0 one of the local var to 0• 00401590 c745fc0a000000 mov dword ptr [ebp-4],0Ah Another local var to 10• 00401597 c745f80f000000 mov dword ptr [ebp-8],0Fh Another local var to 15• 0040159e cc int 3 Our hard coded break point

• 0:000> r• eax=00000000 ebx=00000111 ecx=0013fe74 edx=00000000 esi=00403444 edi=00000001• eip=0040159e esp=0013f860 ebp=0013f874 iopl=0 nv up ei pl nz na pe nc• cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206• Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1+0x1e:• 0040159e cc int 3• 0:000> dds 0013f860 • 0013f860 0013fe74• 0013f864 7e41b788 USER32!GetParent+0x59• 0013f868 00000000• 0013f86c 0000000f• 0013f870 0000000a• 0013f874 0013f884 push current ebp to stack, set new ebp as esp Stack Frame - 1• 0013f878 7c1746cc MFC71!_AfxDispatchCmdMsg+0x3d [f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\cmdtarg.cpp @ 89]• 0013f87c 00403444 Simple_App_Crash3!CSimpleAppCrash3Dlg::messageMap• 0013f880 0013fe74• 0013f884 0013f8b4 push current ebp to stack, set new ebp as esp Stack Frame - 2• 0013f888 7c171915 MFC71!CCmdTarget::OnCmdMsg+0x10a [f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\cmdtarg.cpp @ 396]• 0013f88c 0013fe74• 0013f890 000003e8• 0013f894 00000000• 0013f898 00401580 Simple_App_Crash3!CSimpleAppCrash3Dlg::OnBnClickedButton1 [d:\sisimon\working area\presentations\windbg\simple app

crash3\simple app crash3\simple app crash3dlg.cpp @ 154]• 0013f89c 00000000• 0013f8a0 00000035• 0013f8a4 00000000• 0013f8a8 0013fe74• 0013f8ac 000003e8• 0013f8b0 00000000• 0013f8b4 0013f8d8 push current ebp to stack, set new ebp as esp, Stack Frame - 3• 0013f8b8 7c14db36 MFC71!CDialog::OnCmdMsg+0x1b [f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\dlgcore.cpp @ 88]• 0013f8bc 000003e8• 0013f8c0 00000000• 0013f8c4 00000000• 0013f8c8 00000000• 0013f8cc 000003e8• 0013f8d0 0013fe74• 0013f8d4 00000000• 0013f8d8 0013f928• 0013f8dc 7c175cd8 MFC71!CWnd::OnCommand+0x51 [f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\wincore.cpp @ 2549]

Story 2 – Stack Overflow• WinDBG commands to demo.– dds – dbs– !thread– !irp– lmv– dt– du– .trap– .frame n– dv

Story 3 – Kernel mode Deadlock

• Windbg commands to demo– !locks– !thread– !handle– kb– !irql– !irpfind– ~ <processor>– !devobj– !stacks

Story 4 – Kernel Pool leak

• What is pool tags• WinDBG commands to demo.– !poolused 2– !poolused 4– !vm

Q&A