1. What is Intel L1TF?
The L1TF (L1 Terminal Fault) bug allows an unprivileged attacker to steal privilege data residing in the L1 data cache. It has three variants:
L1 Terminal Fault - SGX：An user-level attacker can steal the Enclave data residing in L1 data cache;
L1 Terminal Fault - OS/SMM：An unprivileged attacker can steal privileged data (belong to either OS or SMM) residing in L1 data cache;
L1 Terminal Fault - VMM：A malicious guest VM (virtual machine) can steal the data belonging to either VMM (virtual machine monitor) or other guest VMs from L1 data cache.
L1TF exists on almost all the Intel CPUs which support speculative execution as well as page table. The official list of the influenced CPUs includes almost all the new Intel CPUs in recent years.
1.1 Background: Page Table, Cache, Speculative Execution and Convert Channel
Page Table: Page table is used to manage memory. When it is enabled, CPU will use virtual address to access the memory. The page table is used to translate the virtual address to physical address which is used to access the physical memory. Each virtual page (commonly 4KB) has its own page table entry, which records its corresponding physical page and permission bits. There are two permission bits related to L1TF, the P-bit (present bit) and reserved bit. These two bits will be checked when the entry is used to perform address translation.
Cache: Cache is used to accelerate memory access. The CPU cache can be divided into three levels (from Level-1 to Level-3). L1 cache (Level-1 cache) is the smallest and fastest one, and can be further divided into L1-data cache and L1-instruction cache. In a multi-core CPU, each CPU core has its own L1 cache.
Speculative Execution: Speculative execution is used to accelerate the instruction execution. When CPU transfers the control flow and cannot immediately gets the new instruction (e.g., indirect jump, interrupt), it will speculatively execute the following instruction. If the speculation is correct, the results are committed and the performance can be improved. Otherwise, the CPU will rollback all its execution results.
Convert Channel: Convert channel is an indirect data transfer method. Two isolated components can control the change of some system status and leverage such change to bypass isolation mechanisms and transfer message. One example is cache convert channel, which leverages cache status (hit or miss) to transfer message.
1.2 Introduction of L1TF
When the CPU accesses the memory, the virtual address will be translated to physical address with the help of page table. During the translation, if the corresponding page table entry is invalid (i.e., the check of the P-bit or the reserved bit fails), the CPU will immediately stop the translation and throw a Page Fault, which is also called “Terminal Fault”.
However, when a terminal fault happens, the CPU may speculatively execute the next few instructions instead of immediately jumping to the fault handler. At this time, the CPU speculatively generates a physical address (according to the content of the page table entry) and continues the execution of the memory access instruction. If the data of the target physical address reside in the L1-data cache, the data will be loaded and passed to the following instructions speculatively.
Finally, all the results of the speculative execution will be rollbacked when the CPU jumps to the fault handler. However, the CPU will not rollback the change of the cache status. Thus, the speculatively executed instruction can build a cache convert channel and pass the data to the attacker.
In the above example, the CPU speculatively generates the physical address to access physical memory after the terminal fault. So accessing this physical address will bypass many memory access checks, including:
Bypass the Enclave memory protection of SGX. Enclave’s data will be encrypted in the memory and only be decrypted in the cache. After the address translation, CPU will check whether the target physical address belongs to current Enclave;
Bypass the memory protection of SMM. SMM (System Manage Mode) is a high privilege mode in Intel CPU. After the address translation, CPU will check whether target physical address belongs to SMM and whether current mode has privilege to access it;
Bypass the address translation of Intel EPT (Extended Page Table). In virtualization environment, the second address translation will be performed using EPT after the first address translation (with page table in guest VM). EPT is widely used to isolate the memory between different VMs.
Thus, L1TF has three variants accordingly: L1TF-SGX, L1TF-OS/SMM and L1TF-VMM.
1.3 Requirements of L1TF
L1TF bug requires that the target data must be loaded into the L1-data cache of the CPU core where the attack code runs. Specifically, L1TF has the following three requirements and all of them need to be satisfied:
The target data must be loaded into the L1-data cache of the CPU core which locates the attack code. Since the size of L1-data cache is limited and the data in it are replaced frequently, the attack window is very small.
The attack code must be located in the CPU whose L1-data cache contains the target data. At the same time, there must be a vulnerable page table entry, which can be used to construct the physical address of the target data.
The attacker can only get the data with speculative execution and convert channel.
The requirement 1 means that the attacker must enforce that the target data are loaded in L1-data cache. Requirement 2 and 3 mean that the attack code must run in the CPU core whose L1-data cache contains the target data. With hyper-threading (HT), one CPU core has two physical threads which can execute instructions concurrently and share the L1-data cache. With HT enabled, the attack code is only needed to be located in one of the two physical threads of the target CPU core.
2. L1TF's impact on Intel SGX
Intel SGX is the new hardware security feature of Intel CPU, which provides a TEE (trusted execution environment) called Enclave. Several new instructions are provided to create, manage and destroy an Enclave. All the code and data within an Enclave are stored in a specific physical memory region called EPC (Enclave Page Cache). Data in EPC are encrypted in memory and decrypted only in cache. The Enclave can not only defend against software-based attack from the privileged mode, but also the physical attack (e.g., cold boot).
2.1 How to leverage L1TF to read the Enclave data?
Although the Enclave data is encrypted in the memory, it is decrypted in the cache. L1TF can bypass the memory access check of Enclave, so that the attacker can steal the Enclave data loaded in L1-data cache. There is a paper, “FORESHADOW: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution“ published on USENIX Security 2018 that contains more details of the attack.
The paper points out that it is easy to leverage the L1TF bug to read the Enclave data, since Intel provides a new instruction eldu, which is used to load Enclave data from normal memory into the EPC. Meanwhile, the eldu also loads the data into the L1-data cache. With eldu, an attacker does not need to guess when the Enclave data will be loaded into L1-data cache. She can directly load the target data into the L1-data cache of current CPU core and then leverage L1TF-SGX to read the data.
2.2 The defense methods provided by Intel
Intel provides the microcode patch to fix the L1TF-SGX bug. The patch includes two defense methods：
Flushing L1-data cache during Enclave enter/exit. It enforces that all Enclave data will not be resided in L1-data cache after Enclave exit, so that the data cannot be stolen with L1TF bug.
Allow Enclave to test whether hyper-threading is enabled. Intel advises user to not trust the Enclave running on the CPU with hyper-threading enabled.
However, Intel does not mention the eldu instruction in its document. We do not know whether the eldu instruction is patched. If it is not,
the attacker may still leverage L1TF to steal the Enclave data.
Intel’s defense method requires user to not trust the Enclave running