In part 1 of my explorations into Hypervisor-Managed Linear Address Translation (HLAT), I installed a Canary build on my AAEON UP Xtreme i12 Alder Lake board, and booted to the Windows desktop to see the VMCS field indicating that HLAT was enabled. This time, I isolated some of the code that actually turns it on.
Firstly, a nod toward Yarden Shafir, whose article, HyperGuard Part 3 – More SKPG Extents, pointed me towards the Windows ShvlSetVpRegister() function, that plays a key role in turning on HLAT.
A lot of the heavy lifting associated with writing this article comes from booting my Alder Lake board from the UEFI shell, with a VM Launch breakpoint set on all processors. You can see the progression of what function is active on each logical processor when hitting VM Launch breakpoints 1 through 24 below:
Note that ShvlpVtl1Entry and SkeStartProcessor are functions within the Secure Kernel, and KeStallExecutionProcessor, KiInitializeKernel and HalpLMStubForVM are in the Normal Kernel.
Determining where and when HLAT is enabled on the target is accomplished by looking at the VMCS_VM_TERTIARY register, bit 1 being set to 1 if HLAT is active:
After some experimentation, I found that HLAT became enabled as of VM Launch break #12 after initial boot. At that point, logical processor P9 has entered the ntkrnlmp!HalpLMStubForVM function. And P8 and P9 have HLAT enabled. The e-cores do not.
But what is happening between VM Launch breakpoint #11, where P9 is in the Secure Kernel, and VM Launch breakpoint #12, where it enters NTOS? This is where some meticulous testing, using Intel Processor Trace, came in.
I knew that HLAT became enabled somewhere upon the second incarnation of ShvlpVtl1Entry (note that that’s on P9, the second logical processor, and not P8; the bootstrap processor already entered ShvlpVtl1Entry back at the third VM Launch breakpoint).
Looking at the TLFS and the Microsoft documentation on HV_REGISTER_NAME (again, thanks, Yarden), I see that the Normal Kernel VTL0 is configured via register HvRegisterVsmVpSecureConfigVtl0 (identifier 0x000D0010). This is done via the nesting of ShvlpVtl1Entry > ShvlpConfigureVtlControls > ShvlSetVpRegister function calls.
So I placed a breakpoint at ShvlpConfigureVtlControls, and then stepped into ShvlSetVpRegister(), noting the RCX (Phase 3) was 3, and R8 was 0x000D0010. Interestingly, the code emerged out of the ShvlSetVpRegister function with the HLAT bit still disabled.
There’s a lot of code between the invocation of ShvlSetVpRegister() and P9 entering the NT kernel. I used a combination of VM Resume breakpoints and Intel PT to see exactly where HLAT was enabled. Among the functions invoked between ShvlSetVpRegister and the entry into the normal kernel are (note that this is not a comprehensive list):
SkiGetCpuVendor SkiDetectHardwareSpeculationFeatures SkiInitializeApic SkiEnableXSave SkiSetRegister SkiApicSettingState
After many hours of rigorous research, I found that the HLAT VMCS bit turns to 1 on a P-core exactly when it enters ntkrnlmp!HalpLMStubForVM.
I’ll look at HLAT being enabled on the e-cores in a future blog.
And many thanks to Andrea Allievi and Satoshi Tanda, who inspired me to research HLAT based upon their excellent 2024 REcon Montreal presentation.
