Tag Archives: hides speculative-execution control mechanism for VMs

PowerCLI script: How to hide the speculative-execution control mechanism for VMs running on affected processors ?

HOME, , , ,

Here is the latest KB update from VMware on Hypervisor-Assisted Guest Mitigation for branch target injection. I would highly recommend you to read this latest KB to understand all minute details. As per KB, it is recommended NOT to apply recently released ESXi patches i.e. ESXi650-201801402-BG, ESXi600-201801402-BG, or ESXi550-201801401-BG on servers based out of affected processors. However, if you have already applied aforementioned patches on affected servers/hosts, VMware recommends to do following in order to hide the speculative-execution control mechanism for virtual machines(quoting from KB)

Update 03/20: As per this KB, new ESXi/vCenter patches are available. With new patches released as on 03/20, this post is NO more valid. Once you start applying new patches, please start using either PowerCLI script for confirming both hypervisor and microcode patches discussed on my earlier blog post or my latest pyVmomi script for the same.

Quote started

For servers using affected Intel processors (see Table 1.) that have applied ESXi650-201801402-BG, ESXi600-201801402-BG, or ESXi550-201801401-BG VMware recommends the following::

On each affected ESXi host, add the following line in the /etc/vmware/config file:
cpuid.7.edx = “—-:00–:—-:—-:—-:—-:—-:—-”
This will hide the speculative-execution control mechanism for virtual machines which are power-cycled afterwards on the ESXi host.
This line will need to be removed after applying a future fixed microcode from Intel in order to enable the full guest OS mitigations for CVE-2017-5715.
When convenient, power-cycle virtual machines on the affected ESXi hosts; rebooting of the ESXi host is not required.

Quote ended

The way I wrote a PowerCLI script for confirming patch validation, I thought it would be really handy to have another quick PowerCLI script to hide the speculative-execution control mechanism for virtual machines as suggested by VMware in KB.

What this script does?

1. Get all the connected hosts from specified vCenter server datacenter.
2. Start iterating through each ESXi host.
3. Check whether ESXi host max supported EVC mode is “intel-haswell” or “intel-broadwell”. If yes, script will then check whether new cpuids are really exposed to these hosts.
4. If new cpuids are exposed on haswell or broadwell hosts, it picks that host for adding “cpuid.7.edx = “—-:00–:—-:—-:—-:—-:—-:—-” into “/etc/vmware/config” file.
5. Before updating above file, script does check whether SSH is enabled on picked (haswell or broadwell) host, if SSH is not enabled, it does enable it.
6. If SSH on picked host is already enabled, it will create SSH session & takes the backup of the file /etc/vmware/config, which gets named as per time stamp.
7. Once file is backed up, it will go ahead and add “cpuid.7.edx = “—-:00–:—-:—-:—-:—-:—-:—-” line at the end of file “/etc/vmware/config”
8. Once file edit operation is done, script does disable SSH on the host where it was not enabled before editing. It also confirms whether SSH is really disabled.
9. Above steps will be repeated only for intel-haswell or intel-broadwell based hosts, where new cpuids are exposed.
10. It keeps logging (on console) the hosts which are edited/not-edited
11. Finally it disconnects vCenter server session.

This script is available on my on my git repo: hideNewCpuids.ps1

After executing above script, it is critical to do VM power-cycle (PowerOFF the VM & PowerON) to get this change affected. Once you do powerCycle, you may wonder how to verify whether speculative-execution control mechanism for virtual machines is actually got hidden or not. When we say “hide the speculative-execution control mechanism for virtual machines”, as earlier blog post pointed, it exposes new CPUIDs to the VM i.e. cpuid.IBRS, cpuid.IBPB, cpuid.STIBP. So it is meant that after script execution, once user does VM power-cycle, these CPUIDs should NO more be exposed to VMs.

How to quickly verify these cpuids are hidden from VM? It can easily done using vCenter MOB (Managed object browser) as follows:

1. To check whether host is having new cpuid capability: please use vCenter MOB (https:// your VC IP/mob): https://10.192.10.20/mob/?moid=host-10&doPath=config (replace your VC IP , host moid and check for “featureCapability” property, you will get list of new cpuids this host is capable of.

Note: Patched ESXi hosts continue to show new CPUIDs (value as 1) even after editing /etc/vmware/config file. This is expected since update to file /etc/vmware/config just hides speculative-execution control mechanism for virtual machines.

2. To check whether new cpuids are hidden from VM:
https://10.192.10.20/mob/?moid=vm-19&doPath=runtime (replace your VC IP , vm moid and check for “featureRequirement” property.

Result:
Before editing file /etc/vmware/config: “”featureRequirement” property of VM shows new CPUIDs i.e. cpuid.IBRS, cpuid.IBPB, cpuid.STIBP

After editing file /etc/vmware/config : “”featureRequirement” property of VM does NOT show new CPUIDs (PowerCycle needed). i.e. cpuid.IBRS, cpuid.IBPB, cpuid.STIBP

How to restore the host to old state? i.e. before editing /etc/vmware/config file
– Before doing this, do make sure that you completely understand the impact it can have on your environment. KB #52345 gives more details.
– To restore, you can either remove added line from /etc/vmware/config or you can comment out using “#” as prefix. To verify this change, you will have to do powercycle. No host reboot required as specified in KB post file update.

Notes:
1. For the sake of simplicity I have hardcoded some values, please do change as per your environment.
2. If you have any comment/feedback on above script, please do provide.
3. I have tested this script on vCenter/ESXi 6.0 & it worked as expected for me.

Additional references:
1. If you are new to MOB, refer official doc

2. I would like to acknowledge this post on Posh-SSH module. I leveraged SSH part of it.

3. If you want to verify above change using PowerCLI, you can leverage nice script written by our own William Lam here.

4. I may write a python script using pyVmomi for verification similar to what we did using MOB above.

I hope this post was useful, please share as appropriate.