On Tuesday, 14th of August Intel has disclosed the new class of CPU speculative-execution vulnerability called “L1 Terminal Fault”. This new class of vulnerabilities can occur on current and past Intel processors (from at least 2009 – 2018) when affected Intel microprocessors are speculating beyond an unpermitted data access. By continuing the speculation in these cases, the affected Intel microprocessors expose a new side-channel for attack, allowing a malicious VM to infer data in the hypervisor and other VM’s running on a core.There are three vulnerabilities have been named:
- CVE-2018-3646 (L1 Terminal Fault – VMM)
- CVE-2018-3620 (L1 Terminal Fault – OS)
- CVE-2018-3615 (L1 Terminal Fault – SGX, SMM) : VMware products does not affect with this
Mitigation of these Vulnerabilities Fall in to two categories :
- Hypervisor -Specific Mitigations
- Operating System Mitigations
This KB provides links to additional KBs with detailed mitigation processes for each of the attack vectors identified by the L1 Terminal Fault vulnerabilities.VMware encourage you to sign up on their Security Announce Mailing List to receive new and updated Security advisories. Protect Your Environment From ( VMSA-2018-0020 ) – Hypervisor Specific Mitigation for L1 Terminal Fault – VMM Vulnerability Make sure to apply patches mentioned in the Security Advisory VMSA-2018-0020 as shown in the below table.
Hypervisor Specific Mitigation Steps
CVE-2018-3646 has two currently known attack vectors for hypervisors which will be referred to here as “Sequential-Context” and “Concurrent-Context.”
Sequential-Context and Concurrent-Context must be addressed to mitigate the CVE-2018-3646
- Sequential-context attack vector: a malicious VM can potentially infer recently accessed L1 data of a previous context (hypervisor thread or other VM thread) on either logical processor of a processor core.
- Concurrent-context attack vector: a malicious VM can potentially infer recently accessed L1 data of a concurrently executing context (hypervisor thread or other VM thread) on the other logical processor of the hyperthreading-enabled processor core.
Sequential-Context Attack Vector Mitigation
Patching the vCenter server and the ESXi hosts to the given versions will mitigate this Security issue and this mitigation is enabled by default and does not impose a significant performance impact.
Patch your vCenter Servers to get the below recommended versions:
- vCenter 6.7.0d
- vCenter 6.5u2c
- vCenter 6.0u3h
- vCenter 5.5u3j
vCenter build versions can be found in this Knowledge Base article. Patch your ESXi Servers to apply the below patches.
- ESXi 6.7
- ESXi670-201808401-BG (esx-base)
- ESXi670-201808402-BG (microcode)
- ESXi670-201808403-BG (esx-ui)
- ESXi 6.5
- ESXi650-201808401-BG (esx-base)
- ESXi650-201808402-BG (microcode)
- ESXi650-201808403-BG (esx-ui)
- ESXi 6.0
- ESXi600-201808401-BG (esx-base)
- ESXi600-201808402-BG (microcode)
- ESXi600-201808403-BG (esx-ui)
- ESXi 5.5
- ESXi550-201808401-BG (esx-base)
- ESXi550-201808402-BG (microcode)
- ESXi550-201808403-BG (esx-ui)
- VMware Workstation Pro 14.1.3
- VMware Fusion Pro 10.1.3
Another tidbit – After applying the L1TF ESXi Patch, you’ll see a “blue” notification message.— William Lam (@lamw) August 16, 2018
This is to indicate there’s also the Side-Channel Aware Mitigation that can be enabled post-L1TF patch. Make sure to go through analysis w/HTAware Mitigation Tool to understand impact pic.twitter.com/7kvaI0SkEX
Note: If you are upgrading the ESXi hosts using Update Manager make sure to download and check whether you are covered with the latest Patches released.
Concurrent-context Attack Vector Mitigation
This mitigation requires a new feature enablement called “ESXi Side-Channel-Aware Scheduler” and please note that this feature may impose a non-trivial performance impact and is not enabled by default. Installing above patches will enable the “ESXi Side-Channel-Aware Scheduler and we need to enable this manually and make sure to think about the performance of your environment before you enable this in your ESXis.
The following identified potential problems are published by VMware:
- VMs configured with vCPUs greater than the physical cores available on the ESXi host
- VMs configured with custom affinity or NUMA settings
- VMs with latency-sensitive configuration
- ESXi hosts with Average CPU Usage greater than 70%
- Hosts with custom CPU resource management options enabled
- HA Clusters where a rolling upgrade will increase Average CPU Usage above 100%
Three Mitigation Phases Have Been Introduced By VMware To Address These Security Contexts
Enable ESXi Side-Channel Aware Scheduler
Using vCenter Server
Login to the vCenter Server and Select the ESXi Host and follow the below Steps to Edit the Parameter ( You have to apply the above patches in order to see these advanced parameters in your ESXi Advanced Parameter List
Enable the VMkernel.Boot.hyperthreadingMitigation parameter
Using ESXi Host Client
Access the ESXi Host Client using your web browser (https://<host_IP_address/ui)
Follow the below steps to enable the VMkernel.Boot.hyperthreadingMitigation Advanced Setting Parameter
Set the Value true
Using esxcli commands
Open a ssh Session to the host and run below command to view the configuration status of the Advanced Setting
esxcli system settings kernel list -o hyperthreadingMitigation
Values should be zero as per the below screen capture and run this command to enable the setting and reboot the host
esxcli system settings kernel set -s hyperthreadingMitigation -v TRUE
Again, Enablement of this scheduler may impose a non-trivial performance impact on applications running in a vSphere environment and planning must be taken in to the consideration prior to enabling these settings.