Last week, I used Architectural Event Trace (AET) to capture all events that invoked Model Specific Register (MSR) reads and writes. This week, I use the Trace Hub to trace Intel Management Engine (ME, also known as Converged Security and Management Engine (CSME)) events.
As a refresher, last week we halted the Skylake server target in the middle of some random code while the CPU was in Protected mode, and captured all RDMSR and WRMSR instructions using AET:
And the Event Trace window (expanded for a better look) showed these:
This is extremely useful – it would be impossible to otherwise know where these reads and writes were happening in an effective way. Sure, you could use breakpoints if you knew what you were looking for, but you’ll quickly run out of hardware breakpoints, the results wouldn’t be timestamped, it would be extremely tedious, and you wouldn’t get the big picture.
Now, let’s look at how to use the Intel Trace Hub (ITH) to track ME messages. Note that we’re not using AET here (but we will; that’s for later). The Master definitions in SourcePoint for ME Message Trace are 10-12:
And again, let’s trace using the Master Storage Controller Trace Buffer (MTB), as opposed to system memory, so that we can capture activity from early boot.
It’s that simple. We reset the target and halt in PEI. Here are the messages captured:
This is pretty exciting, right? Tracing the ME activity is crucial as it represents a separate execution engine versus what is running on the CPU cores, and it could be guilty of errantly modifying objects in the host memory space.
The status codes, states and arguments are obfuscated somewhat, and you’d approach Intel for some clarity on these if needed. But, finding any sort of pattern when troubleshooting the most obscure bugs, and maybe finding the “smoking gun” that helps identify root cause, can be a joy!
Note also that there are UEFI messages interleaved with the ME messages in the trace buffer. This uses a facility whereby printf instructions are redirected through the trace hub, as opposed to going out the serial port. And you can see that these messages are timestamped, so you see them in the overall execution flow and see the exact sequence of events as the ME may interact with things happening in the BIOS.
And of course other events, including AET events as shown in my blogs Intel Architectural Event Trace, Intel AET in action, and Advanced Capabilities of AET can be interleaved within the Trace Hub and displayed in SourcePoint, and all presented in a meaningful code context. Very nice.
For those who might be interested in my journey learning x86 architecture, UEFI, the Linux kernel, and other topics of related interest, I direct you to my eBook, The MinnowBoard Chronicles Volume 1 (note: requires registration). And links to all 45 chapters are here: The MinnowBoard Chronicles: A Journey into x86, UEFI and Linux.
