TL;DR After noticing a trend of conference talks with posted slides or videos being overlooked and hard to search through, I've decided to write an overview blog post for most of my conference talks in order to create a searchable and easy-to-skim jumping-off point. I will link to slides, video and paper if there is one at the end of each post.

This post is about a neat consequence of PCIe being misused by the CPU in order to control the page tables.

Cluck Cluck

This talk presented a neat trick I found when I was trying to "break out" of virtual memory as kernel shellcode. While I had ring 0 privileges, I did not know where in memory I was operating, and unable to map to arbitrary memory regions without a pointer to the page tables. The catch-22 with this is that you need to know the virtual address (VA) of the page tables in order to map in/out arbitrary physical memory, but you can only find (architecturally at least) the physical address (PA) of the page tables.

Now, if I was just trying to solve this problem without doing anything clever, I'd have remembered that Windows (at least back in 2010 when I did this work) typically has a mapping to the page tables at VA 0xC0000000. However, finding an OS-independent way to do this was much more up my alley. The crux of this problem is that without being able to modify memory at a known PA, you cannot set up a recursive mapping to the page tables. If I could gain control over only 32-bits of physical memory and know where it was, I could map in the OS page tables and I'd have done it.

PCIe Extended Configuration Access Mechanism (ECAM)

Remembering back to the halcyon days of PCI configuration, performed by reading and writing to ports 0xCF8 and 0xCFC, I also had learned that when the PCIe specification came out, the PCI SIG wanted to expand the size of configuration space, and provide a faster method to access and configure devices. Thus the ECAM was born, in which the memory controller hub (MCH) shadows in PCI configuration space into physical memory, providing the OS with the ability to manage the PCI device tree with memory-mapped IO (MMIO). The MCH helpfully stores the base of this memory shadowing location in a register (PCIEXPBAR) in its PCI configuration space, which can still be accessed via port IO (D0:F0 Offset: 0x60).

Now that I could find the physical memory location for PCI configuration space, and modify it via port IO, all I needed was a configuration space register that would be on most/all systems I came across and wouldn't cause system instability if it was modified to an arbitrary value. Thankfully, Intel provides the scratch-pad data (SKPD) register on its MCHs (D0:F0 Offset:0xDC), this register is specifically designed to do nothing and allow software to write data to it. Below is a picture to explain what I was trying to do:

The "Hack"

With the SKPD register shadowed into physical memory at a known location, I could create a temporary page table mapping the OS page tables into memory, then create a more permanent recursive mapping. This entailed reading the PA of the page table base from CR3 and creating a custom page directory entry (PDE) with PS = 1 (large pages) that mapped in the OS page table base. I would put this entry into the SKPD register and calculate the VA to account for the alignment requirements of the page table base in CR3.

Once my mini page table was set up, I could save off the old CR3 and replace it with mine in the ECAM. Then walk through the OS page tables with this temporary mapping I had created, looking for an empty entry to map in the OS page tables at a known VA. Then CR3 is simply restored to the OS, and my shell code had a pointer to the page tables with which it could map in arbitrary physical memory or MMIO regions.

Conclusions

While this is not a security vulnerability in the sense that is requires elevated privileges to begin with, it is a neat "weird machine" in the architecture and I found it helpful for my use-case. Upon further revisions, I found that I could use ATA DMA from ring 3 as long as I had IO privileges (e.g. root access or iopl()) by using the ECAM as a configuration table for the HDD to write to arbitrary physical memory. The lesson of this is to watch for "complexity creep"and to not run software with IO privileges.

• Timeline: work performed in 8/2010, presented 8/2014
• Slides
• Video