AMD-V for Hackers - GitHub Pages
Transcript of AMD-V for Hackers - GitHub Pages
![Page 1: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/1.jpg)
AMD-V for
HackersSATOSHI TANDA (@STANDA_T )
1
![Page 2: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/2.jpg)
Concept of Stealth Hooking
Hooking: Modify code and execute arbitrary logic
instead of, or on the top of original logic
Allows us monitoring and altering behavior of
software
Very useful for reverse engineering software
Some detects that hooks are installed
PatchGuard, protectors
Stealth: Make them invisible from the target, using
SLAT
2
![Page 3: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/3.jpg)
Who I am
Software Engineer at CrowdStrike
Reverse Engineer
Twitter: @standa_t
Speaker: Recon, BlueHat, Nullcon, CodeBlue
Hiker, camper, runner, diver, cat lover, husband
etc
Creator of HyperPlatform – an open source
hypervisor on Windows
3
![Page 4: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/4.jpg)
Motivation
4
![Page 5: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/5.jpg)
Hypervisors are…good
Powerful
Interesting
Manageable
5
![Page 6: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/6.jpg)
Background
HyperPlatform was created for the community
DdiMon, MemoryMon, GuardMon
Lots of adoption and interests
SimpleSvm was written for AMD users
Lack of implementation of the useful ideas
Can we have the feature parity with Intel VT-x
based hypervisors?
6
![Page 7: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/7.jpg)
Basics of HW VT for
Hackers
7
![Page 8: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/8.jpg)
Hypervisor as a tool
Just a normal Windows driver (.sys)
Can be developed with Visual Studio, tested in a
VMware
8
Hardware Hardware
Host OS Host OS Hypervisor
Hypervisor
Guest OS Guest OS
Virtualized
Virtualized
Type-2 Hypervisor Hyperjack-type Hypervisor
![Page 9: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/9.jpg)
Hypervisor as a tool
Once loaded, the driver enables hardware-based
virtualization technology (HW VT) of the processors;
Set EFER_SVME to IA32_MSR_EFER (Intel: VMXON)
Sets up a context structure VMCB (Intel: VMCS); and
Starts virtualization of the processors
VMRUN (Intel: VMLANCH)
Processors trigger #VMEXIT (Intel: VM-exit) where your
hypervisor handles
9
![Page 10: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/10.jpg)
Hypervisor as a tool 10
User-Mode
Kernel-Mode
Kernel YourHv.sys
1) Set EFER_SVME
DriverDriver
Driver
ApplicationApplication
Application
![Page 11: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/11.jpg)
Hypervisor as a tool 11
User-Mode
Kernel-Mode
KernelDriver
DriverDriver
YourHv.sys
ApplicationApplication
Application
3) #VMEXIT
2) Memory access
2) MOV CR3, …2) Exceptions
![Page 12: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/12.jpg)
Use cases
System-wide debugger
Cheat Engine:DBVM
PatchGuard, rootkit analysis
HyperBone
Vulnerability hunting?
12
![Page 13: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/13.jpg)
SLAT: Second Level
Address Translation
AKA, MEMORY VIRTUALIZATION
13
![Page 14: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/14.jpg)
What SLAT is
SLAT allows a hypervisor to intercept and alter
translation of virtual memory address (VA) to
physical memory address (PA)
An architecture agnostic term
Called Rapid Virtualization Indexing (RVI), or Nested Page Tables (NPT)
Intel: Extended Page Tables (EPT)
14
![Page 15: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/15.jpg)
How SLAT works
1. The processor performs the regular VA->PA
translation with CR3 and page table structures
2. That PA is called Guest PA (GPA)
3. The processor performs GPA -> System PA (SPA)
with nCR3 and NPTs
Intel: with the EPT pointer and EPTs
15
![Page 16: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/16.jpg)
Address Translation w/o VT 16
Linear Address
CR3 & Page Tables
Physical Address
![Page 17: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/17.jpg)
Address Translation w/ VT
Read/Write/Execute permissions can be configured like the regular PTEs
17
Linear Address
CR3 & Page Tables
GPA
GPA
nCR3 & Nested
Page Tables
System Physical Address
![Page 18: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/18.jpg)
Stealth Hooking with SLAT
18
![Page 19: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/19.jpg)
Concept
Hooking: Modify code and execute arbitrary logic
instead of, or on the top of original logic
Allows us monitoring and altering behavior of
software
Very useful for reverse engineering software
Some detects that hooks are installed
PatchGuard, protectors
Stealth: Make them invisible from the target, using
SLAT
19
![Page 20: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/20.jpg)
Implementation
1. Set up two SLAT translations for the target GPA
Exec: translates to the SPA with hook
Not readable, not writable but executable (--X)
ReadWrite: translates to the SPA without hook
Readable, writable, but not executable (RW-)
2. At the runtime
On execute access, use Exec translation
The hook is executed
On read and write access, use ReadWrite translation
The hook is not visible and overwritable
20
![Page 21: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/21.jpg)
Implementation on Intel
Assume ZwQuerySystemInformation is at
0xfffff800`00000000 (VA) => 0xe000 (GPA)
Hypervisor creates two translations:
Reading ZwQuerySystemInformation does not
reveal hook
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 --X With hook
21
![Page 22: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/22.jpg)
Implementation on Intel
On execution, VM-exit occurs due to access
violation
Hypervisor switches translation and let the processor
retry the same execution
Hook is executed
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 --X With hook
22
![Page 23: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/23.jpg)
Implementation on Intel
On read, VM-exit occurs due to access violation
Hypervisor switches translation and let the processor
retry the same execution
ZwQuerySystemInformation is read without hook
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 --X With hook
23
![Page 24: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/24.jpg)
On AMD
24
![Page 25: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/25.jpg)
Challenges on AMD
Not possible to set the execute-only permission
To be executable, the page must be readable too
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 RWX With hook
25
![Page 26: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/26.jpg)
Challenges on AMD
On execution, #VMEXIT occurs due to access
violation
Hypervisor switches translation and let the processor
retry the same execution
Hook is executed
The hook remains visible!
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 RWX With hook
26
![Page 27: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/27.jpg)
Solution?
Trigger #VMEXIT and switch translation to the non-
executable version to hide the hook ASAP?
Eg, set eflags.TF, let the hypervisor handle it and
switch translation?
Performance impact is significant
Consider when the processor want to execute
thousands of instructions on 0xe000
Every single instruction execution, access violation
and eflags.TF trigger #VMEXIT
27
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 RWX With hook
![Page 28: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/28.jpg)
Partial Solution
The page must remain executable and readable
for performance
… while the page is being executed
Switch translation when the processor goes
outside the page
28
GPA SPA Permission Memory Contents
0xe000 0xe000 RW- Without hook
0xe000 0xf000 RWX With hook
![Page 29: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/29.jpg)
Partial Solution
Make all other pages non-executable while the
hooked page is executed
29
GPA SPA Permission
0x0 0x0 RW-
0x1000 0x1000 RW-
0x2000 0x2000 RW-
... ... ...
0xe000 0xf000 RWX
... ... ...
0x8000`0000 0x8000`0000 RW-
![Page 30: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/30.jpg)
Partial Solution
Make all other pages non-executable while the
hooked page is executed
30
GPA SPA Permission
0x0 0x0 RW-
0x1000 0x1000 RW-
0x2000 0x2000 RW-
... ... ...
0xe000 0xf000 RWX
... ... ...
0x8000`0000 0x8000`0000 RW-
When the processor
go outside the page,
#VMEXIT occurs
Hypervisor restores
permissions of the
pages
![Page 31: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/31.jpg)
Partial Solution
Make all other pages non-executable while the
hooked page is executed
31
GPA SPA Permission
0x0 0x0 RWX
0x1000 0x1000 RWX
0x2000 0x2000 RWX
... ... ...
0xe000 0xf000 RWX
... ... ...
0x8000`0000 0x8000`0000 RWX
When the processor
go outside the page,
#VMEXIT occurs
Hypervisor restores
permissions of the
pages
![Page 32: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/32.jpg)
Partial Solution
Make all other pages non-executable while the
hooked page is executed
32
GPA SPA Permission
0x0 0x0 RWX
0x1000 0x1000 RWX
0x2000 0x2000 RWX
... ... ...
0xe000 0xe000 RW-
... ... ...
0x8000`0000 0x8000`0000 RWX
When the processor
go outside the page,
#VMEXIT occurs
Hypervisor restores
permissions of the
pages; and,
Hide the hook
![Page 33: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/33.jpg)
Limitation
Hook remains visible from code in the same page
Further partial solution for the limitation: as soon as hook is executed, switch to the page withouthook (ie, even before jumping out to the other page). Downside is that hook is not executed from code in the same page + little more perf cost due #VMEXIT for page switching.
33
GPA SPA Permission
0x0 0x0 RW-
0x1000 0x1000 RW-
0x2000 0x2000 RW-
... ... ...
0xe000 0xf000 RWX
... ... ...
0x8000`0000 0x8000`0000 RW-
![Page 34: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/34.jpg)
Performance Issue
34
![Page 35: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/35.jpg)
Naïve Implementation
524,288 -1 NTP entries must be updated to change
permission of all pages on a system with 2GB RAM
2GB = 0x8000`0000 = 0x8000`0000 / 0x1000 = 0x8`0000
Significant performance impact
Protip: Measure performance. ALWAYS.
35
![Page 36: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/36.jpg)
Optimization 1
Use the nesting NPT structures
Like the regular page structures, NTP is made up of
PML4, PDP, PD, PT
Updating the permission on PML4 entry changes the
permission of entire 512GB
PDP=1GB, PD=2MB, PT=4KB
Update the upper level NTP entries where possible
36
![Page 37: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/37.jpg)
Optimization 2
Enable optimization of NTP manipulation code even on Debug build
Tip: such pure algorithmic code can be tested with an UM project separately
37
![Page 38: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/38.jpg)
Optimization 3
Keep the number of hooks one on VMware
NTP manipulation on VMware is very (VERY) slow
Performance measurement on VMware is pointless
To measure performance and impact of
optimization, use a bare metal machine
38
![Page 39: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/39.jpg)
Demo
39
![Page 40: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/40.jpg)
Conclusion
40
![Page 41: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/41.jpg)
Takeaways
Implementation of stealth hooking on AMD
processors are possible
But beware of the limitations:
Hooks remain visible from code in the same page
Or hooks are not called from code in the same page
Performance overhead is higher than similar
implementation for Intel processors due to extensive
manipulation of NTP entries
Any performance measurement must be done on
bare metal
41
![Page 42: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/42.jpg)
Resources
Learn more about writing hypervisors on AMD
https://github.com/tandasat/SimpleSvm
And implementation of stealth hooking
https://github.com/tandasat/SimpleSvmHook
42
![Page 43: AMD-V for Hackers - GitHub Pages](https://reader031.fdocuments.us/reader031/viewer/2022012513/618c29f9a08add46923101bc/html5/thumbnails/43.jpg)
Thank youFOR VXCON CREW, AND ALL OF YOU LISTENING <3
43