Category Archives: vSphere API

posts on vSphere API/SDK samples/scripts

Simple workaround: Adding a host to an empty EVC cluster fails on latest vCenter patch

This article is motivated by this VMTN thread posted by one of VMware users. I see that several users have reported this usability issue. Issue reported was: Users are NOT able to add a host into an empty EVC cluster after applying latest vCenter patch, which was released as part of Hypervisor-Assisted Guest Mitigation. In this post, I thought to share my understanding about this “empty EVC cluster” behavior along with simple workaround to fix this usability issue. In order to explore this further, I created a empty EVC cluster on vCenter 6.5 U1e as shown below.

You can see, I have created empty EVC cluster, where EVC mode configured is “Intel-IvyBridge”. Now I tried adding a host, where corresponding ESXi patch is NOT applied and as posted on VMTN thread, I see below error on web client.

Now question is why does it fail with this latest patch? You might know that as part of VMware patches released for Hypervisor-Assisted Guest Mitigation, there are three new cpuids got exposed i.e. cpuid.IBRS , cpuid.IBPB & cpuid.STIBP. Now we need to know whether these new cpuids are exposed by the “empty EVC cluster” that we just created but you might be wondering how to go about it? It is fairly simple to know it by using one of EVC API properties as posted in my tutorial post on EVC vSphere APIs. As part of section #4 of that tutorial i.e. “Exploring EVC Cluster state”, I had written a pyVmomi script to get the EVC cluster state i.e. getEVCClusterState.py. I little modified this script to know whether new cpuids are exposed by cluster or not. Below is the code snippet.

#Cluster object for empty EVC cluster
cluster = get_obj(content,[vim.ClusterComputeResource],cluster_name)
#EVCManager which exposes EVC APIs
evc_cluster_manager=cluster.EvcManager()

evc_state=evc_cluster_manager.evcState
current_evcmode_key= evc_state.currentEVCModeKey

if(current_evcmode_key):
        print "Current EVC Mode::"+current_evcmode_key
else:
        print "EVC is NOT enabled on the cluster"
        quit()

#getting feature capabilities available on cluster
feature_capabilities = evc_state.featureCapability

flag=False
for capability in feature_capabilities:

        if(capability.key=="cpuid.STIBP" and capability.value=="1"):
                print "Found new cpuid::"+ capability.key
                flag=True

        if(capability.key=="cpuid.IBPB" and capability.value=="1"):
                print "Found new cpuid::"+ capability.key
                flag=True

        if(capability.key=="cpuid.IBRS" and capability.value=="1"):
                print "Found new cpuid::"+ capability.key
                flag=True
if(not flag):
        print "No new cpubit found"

Once I executed above script on “empty EVC cluster” and I see output as follows

Output:
vmware@localhost:~$ python emptyEVCCluster.py
Current EVC Mode::intel-ivybridge
Found new cpuid::cpuid.IBPB
Found new cpuid::cpuid.IBRS
Found new cpuid::cpuid.STIBP
vmware@localhost:~$

Output clearly shows that new cpuids are exposed on the “empty EVC cluster” that we created on latest vCenter patch. This concludes that when you try to add a non-patched host into this cluster, it will fail since that host does NOT support these new cpuids. On the other hand, if you have applied corresponding ESXi patch (which is pulled back at the moment), then you will not face this issue as new cpuids are available on that patched host. Now that you understood this behavior, let us take a look at the simple workaround to fix this issue.

What is the workaround?
I understand, this is a problem from usability perspective but good news is that we have very simple workaround available i.e. Instead of upfront enabling EVC on the empty cluster, you can create non-EVC cluster first , add all possible hosts into it and finally enable EVC as per available EVC mode. If you ask me, it is always good idea to create non-EVC cluster and only after adding all possible hosts, we should finally enable EVC on cluster.

I would also recommend you to read “vMotion and EVC Information” section from VMware KB to understand EVC and vMotion behavior post applying these patches.

Some useful resources on EVC
1. Part-1: Managing EVC using pyVmomi
2. Part 2: Managing EVC using pyVmomi
3. Tutorial on getting started pyVmomi

I hope you enjoyed this post, let me know if you have any questions/doubts.

Tutorial: Part-2 : How to Manage EVC using vSphere Python SDK pyVmomi?

In part-1, we explored EVC enable, disable APIs & some important EVC properties using pyVmomi. In part-2, we would explore left out EVC APIs from part-1 i.e.CheckConfigureEvcMode_Task() & CheckAddHostEvc_Task(). In addition, we will also take a look at some more important EVC properties required to manage EVC end to end.

1. Checking whether EVC can be configured on existing cluster or not?
CheckConfigureEvcMode_Task() is responsible for checking exactly the same. If you have existing cluster with several hosts and if you would like to evaluate whether EVC can be enabled on cluster or not on particular EVCMode key, this API is really handy. If it is not possible to enable EVC on particular cluster, this API will return not only the set of reasons why it can fail to configure EVC but also the set of hosts those are causing it to fail. Before we start, below are the set of hosts present in cluster already.

vmware@localhost:~$ python getMaxEVCModeonHost.py
ClusterName::vThinkBVMCluster
10.160.20.30::intel-broadwell
10.160.20.31::intel-sandybridge
vmware@localhost:~$

To start with, the way we did in part-1, we will check whether EVC can be enabled on “intel-broadwell” EVCMode key. As we learned in last post, EVC should not be able to configure on this cluster with EVCMode “intel-broadwell”. Let us see whether API properly returns right reason and host that is causing it.

Below script is available on my git-hub repo as well

from pyVim.connect import SmartConnect, Disconnect
from pyVmomi import vim
import atexit
import ssl
import sys
import time

#Script to check whether EVC can be enabled on cluster with given EVCMode

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.160.50.60", user="Administrator@vsphere.local", pwd="VMware#12",sslContext=s)
content=si.content
cluster_name="vThinkBVMCluster";

# Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
        obj = None
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for c in container.view:
                if name:
                        if c.name == name:
                                obj = c
                                break
                        else:
                                obj = None                          
        return obj

#Cluster object
cluster = get_obj(content,[vim.ClusterComputeResource],cluster_name)
evc_cluster_manager=cluster.EvcManager()

print "ClusterName::"+cluster.name

task=evc_cluster_manager.CheckConfigureEvcMode_Task("intel-broadwell")
time.sleep(5)
checkResult= task.info.result

if(checkResult):
         print "EVC can not be enabled on this cluster, please take look at below reasons and hosts causing this issue"
        for result in checkResult:
                print result.error.msg
                for h in result.host:
                        print h.name
                print "---------------------"
else:
        print "EVC can be successfully enabled on the cluster with no issues"

atexit.register(Disconnect, si)

vmware@localhost:~$ python checkEVC.py
ClusterName::vThinkBVMCluster
EVC can not be enabled on this cluster, please take look at below reasons and hosts causing this issue
The host’s CPU hardware does not support the cluster’s current Enhanced vMotion Compatibility mode. The host CPU lacks features required by that mode.
10.160.20.31
———————

isn’t this error matching with this screenshot from Part-1?

Note: If you notice on line #38, CheckConfigureEvcMode_Task() API returns ClusterEVCManagerCheckResult only when EVC can NOT be configured on the cluster.

Now I did powerON one of the VMs on host, which has max EVCMode as “intel-broadwell” and executed above script but this time passed EVCMode key as “intel-sandybridge”. Let us take a look at output.

Output:
vmware@localhost:~$ python checkEVC.py
ClusterName::vThinkBVMCluster
EVC can not be enabled on this cluster, please take look at below reasons and hosts causing this issue
The host cannot be admitted to the cluster’s current Enhanced vMotion Compatibility mode. Powered-on or suspended virtual machines on the host may be using CPU features hidden by that mode.
10.160.20.30
———————
vmware@localhost:~$

You can see, EVC configuration failed since one of VM was powered ON, which is using some cpu features those are NOT available on “intel-sandybridge” EVC mode. This is expected and error is exactly matching with web client screenshot from Part-1.

Now I powered-off the VM residing on “intel-broadwell” based host & Powered-ON VM on host, which support max EVCMOde as “intel-sandybridge”. Finally executed same checkEVC.py script and below is the output.

Output:
vmware@localhost:~$ python checkEVC.py
ClusterName::vThinkBVMCluster
EVC can be succesfully enabled on the cluster with no issues
vmware@localhost:~$

wasn’t output cool? Note that VM was up on host with “intel-sandybridge” host and EVCMode used was also “intel-sandybridge”, hence EVC will be configured successfully. Now let us move on to next EVC API.

2. Checking whether host can be added into existing EVC enabled cluster or not?
CheckAddHostEvc_Task() API allows user to check exactly the same. This API accepts HostSpec as paramter for the host to be checked and below is the code snippet. I will leave it to you in order to try this API in your environment.

host_conn_spec=vim.host.ConnectSpec()
host_conn_spec.force=True
host_conn_spec.hostName=host.name
host_conn_spec.userName=host_user
host_conn_spec.password=host_pass
thumb_print=host.summary.config.sslThumbprint
host_conn_spec.sslThumbprint=thumb_print
print host_conn_spec
task=evc_cluster_manager.CheckAddHostEvc_Task(host_conn_spec)

That is all about EVC APIs. I hope Part-1 & Part-2 have given you good insight into how to play around EVC APIs. Stay tuned for my next post. Please comment if you have any query.

Tutorial: How to manage Enhanced vMotion Compatibility (EVC) using vSphere Python SDK pyVmomi? Part-I

As part of recent security issues i.e. Spectre & Meltdown, I got an opportunity to play around  one of famous vSphere features i.e. Enhanced vMotion Compatibility (EVC) and  I thought this is the right time to explore EVC vSphere APIs . I see that EVC APIs are made public as part of vSphere 6.0 release though feature itself was introduced in vSphere 4.0. Since there is lot to learn about EVC vSphere APIs, I thought it is better to divide this topic in 2 parts. These parts will be kind of tutorial on EVC APIs and at the same time, we will touch upon EVC functionality as well. Before we start exploring EVC APIs, it is important that you understand what EVC is all about & why one needs to enable it on cluster? For quick learning, I think, no better resource than this EVC FAQ KB. In short, EVC makes it possible to mix different generations of CPU servers inside a cluster, which enables vMotion across these servers. Once EVC is enabled on vSphere cluster level, it ensures that all the servers across different CPU generation exposes same cpuid bits to its virtual machines.

When it comes to EVC APIs, ClusterEVCManager is the major managed object, which exposes 4 important methods as follows.

  •  ConfigureEvcMode_Task()
  •  DisableEvcMode_Task()
  • CheckConfigureEvcMode_Task()
  • CheckAddHostEvc_Task()

1. Getting max EVCMode key

Before we look into above methods, we need to get max EVCMode supported by each ESXi host to be added into EVC enabled cluster. Let us take a look at simple pyVmomi script for the same.

Below script is available on my github repo getMaxEVCMode.py

#Script to get Max EVC Mode supported on all the hosts in the cluster

from pyVim.connect import SmartConnect, Disconnect
from pyVmomi import vim
import atexit
import ssl
import sys

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.160.50.60", user="Administrator@vsphere.local", pwd="VMware#12",sslContext=s)
content=si.content

# Your cluster name
cluster_name="vThinkBVMCluster"

# Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
        obj = None
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for c in container.view:
                if name:
                        if c.name == name:
                                obj = c
                                break
                        else:
                                obj = None
        return obj

#Cluster object
cluster = get_obj(content,[vim.ClusterComputeResource],cluster_name)
print "ClusterName::"+cluster.name

# Get all the hosts available inside cluster
hosts = cluster.host

#Iterate through each host to get MaxEVC mode supported on the host
for host in hosts:

        host_max_evcmode = host.summary.maxEVCModeKey

        if host_max_evcmode == None:
                print host.name+" does not support EVC"
        else:
                print host.name+"::"+host_max_evcmode

#Disconnect the vCenter server session.
atexit.register(Disconnect, si)

Above script takes cluster name as input, get all the hosts inside the cluster and iterate through each host in order to get max EVCMode required to enable the EVC on right mode. Line #37 is what gets us “maxEVCModeKey”. Let us take look at how output looks like

Output :
vmware@localhost:~$ python getMaxEVCModeonHost.py
ClusterName::vThinkBVMCluster
10.160.20.30::intel-broadwell
10.160.20.31::intel-sandybridge
vmware@localhost:~$

You could see, there are 2 hosts inside the cluster and their max EVCMode keys are intel-broadwell and intel-sandybridge respectively. We need to choose right max EVCMode key to enable EVC on the cluster.

You may be wondering, can EVC modes supported per host be viewed from vSphere web client? Yes, it is, please take a look at below screenshot.

2. Enable EVC on cluster

Now that we have got max EVC mode per host, let us take a look at EVC API responsible to enable EVC on the cluster. i.e. ConfigureEvcMode_Task(). This API require EVC mode to be passed as parameter.

To start with, we will pass max EVCMode as “intel-broadwell” and see what happens. Lets take a look at script below.

Below script is available on my github repo enableEVC.py

from pyVim.connect import SmartConnect, Disconnect
from pyVmomi import vim
import atexit
import ssl
import sys

#Script to enable/disable EVC on cluster

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.160.50.60", user="Administrator@vsphere.local", pwd="VMware#12",sslContext=s)
content=si.content

#Your cluster input
cluster_name="vThinkBVMCluster"

# Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
        obj = None
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for c in container.view:
                if name:
                        if c.name == name:
                                obj = c
                                break
                        else:
                                obj = None
        return obj

#Cluster object
cluster = get_obj(content,[vim.ClusterComputeResource],cluster_name)

# Major EVC cluster manager object
evc_cluster_manager=cluster.EvcManager()

print "ClusterName::"+cluster.name

# Configure EVC mode by passing EVCMode
task=evc_cluster_manager.ConfigureEvcMode_Task("intel-broadwell")

atexit.register(Disconnect, si)

Notice the line #40, where method is called to enable EVC. Right after executing above script, I looked into web client to see whether EVC is really enabled or not and below is what I observed.

Error displayed above is expected since we are enabling EVC on cluster with “intel-broadwell” EVCMode, which has additional cpuid bits (CPU feature set) those are NOT available on other host inside the cluster. Since “intel-broadwell” is latest intel CPU model, “intel-sandybridge” host does not have new features available. In other words, host with “intel-sandybridge” EVCMode has all the cpu features are available on “intel-broadwell” based host but not the vice-versa, hence we can only enable EVC on cluster with “intel-sandybridge or lower EVCMode.

Now before I execute above script with “intel-sandybridge” EVCMode, I powered on all the VMs available inside the cluster, once VMs are powered ON, I executed above script by editing EVCMode as “intel-sandybridge” on line #40. Since “intel-sandybridge” cpu feature set is available on “intel-broadwell”, you might be hoping that EVC will be successfully enabled on cluster. Let us take a look at web client screenshot.

You could see EVC is NOT enabled again. You might be wondering why is EVC not enabled even though EVCMode is right? As error states, since cluster has Powered-ON VMs those can potentially be using CPU features, which can not be available after enabling EVC on “intel-sandybridge” mode. Reason is EVC will mask additional CPU features (from broadwell host) to have same CPU features across all the hosts inside cluster. I hope you are following.

Now I powered-off and executed the script to enable EVC on “intel-sandybridge” mode. Let us take a look at below screenshot.

You can see EVC is enabled successfully, isn’t it cool?

Note: It is recommended to enable EVC on cluster with min of all hosts maxEVCMode key Ex. min (intel-sandybridge, intel-broadwell) = intel-sandybridge

3. Disabling EVC on cluster

Disabling EVC on cluster is fairly simple using DisableEvcMode_Task() , we have to just replace line #40 with below line.

task=evc_cluster_manager.DisableEvcMode_Task()

After making change as specified above, I confirmed from web client that EVC is successfully disabled. Please take a look at below screenshot.

EVC disabled

You can use Task object returned by ConfigureEvcMode_Task() & DisableEvcMode_Task() methods for tracking the task progress, any errors, state etc.

Now that you learned how to configure, enable & disable EVC on cluster, let us now learn how to know what cpu features are masked by EVC, what cpu features are available on each hosts inside EVC enabled cluster. If you ask me, again it is very simple.

4. Exploring EVC Cluster state

How to get current EVCMode enabled on cluster ? How to know whether EVC is disabled on cluster or not? How to get cpu features available & features masked on EVC cluster? All these questions are answered as part of below script.

Below script available on my github repo getEVCClusterState.py

from pyVim.connect import SmartConnect, Disconnect
from pyVmomi import vim
import atexit
import ssl
import sys
#Script to get Max EVC Mode supported on all the hosts in the cluster
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.160.50.60", user="Administrator@vsphere.local", pwd="VMware#12",sslContext=s)
content=si.content
cluster_name="vThinkBVMCluster"

# Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
        obj = None
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for c in container.view:
                if name:
                        if c.name == name:
                                obj = c
                                break
                        else:
                                obj = None
        return obj

#Cluster object
cluster = get_obj(content,[vim.ClusterComputeResource],cluster_name)
evc_cluster_manager=cluster.EvcManager()

evc_state=evc_cluster_manager.evcState
current_evcmode_key= evc_state.currentEVCModeKey

if(current_evcmode_key):
        print "Current EVC Mode::"+current_evcmode_key
else:
        print "EVC is NOT enabled on the cluster"
        quit()

feature_capabilities = evc_state.featureCapability

for capability in feature_capabilities:
        print "Feature Capability\n"
        print capability.featureName
        print capability.key
        print capability.value
        print "-------------"

features_masked = evc_state.featureMask

for mask in features_masked:
        print "Feature Masked\n"
        print mask.featureName
        print mask.key
        print mask.value
        print "-------------"
atexit.register(Disconnect, si)

That is all for Part-1, I hope you enjoyed.

Update : Part-2 is ready as well, where we will explore remaining EVC APIs i.e. CheckConfigureEvcMode_Task() & CheckAddHostEvc_Task().

For more learning, EVC official documentation is here

How to deploy vCenter HA using pyVmomi?

In part 1, we explored new vCenter HA APIs for managing vCenter HA. In part 2, we will focus on how to deploy vCenter HA using pyVmomi. As I pointed in part-1, managed object FailoverClusterConfigurator is responsible for deploying vCenter HA & below are the methods exposed by this managed object.

  • deployVcha_Task
  • configureVcha_Task
  • createPassiveNode_Task
  • createWitnessNode_Task
  • destroyVcha_Task

Let us start with first and important method i.e.  deployVcha_Task() : This method is really handy when user wants to deploy as well as configure vCenter HA in single call. I will encourage you to read vSphere API reference for more details on this method. This method assumes that you already deployed vCenter server,which would act as active node and created required vCenter HA network. As I wanted to deploy vCenter HA in basic mode, I created my self managed vCenter server as shown below & configured required vCenter HA network.

Before you start deploying your active node, please take a look at vCenter HA software & hardware requirement. I would also recommend you to just overview this & this documentation. If you need any help on deploying active node, please let me know, I would be happy to help you.

Lets take a look at code now. Note that detailed code documentation is added inside script itself for your easy reference & all below scripts are available on my github repo as well.


from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
# Deploying vCenter HA in basic mode using self managed VC 

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.161.34.35", user="Administrator@vsphere.local", pwd="VMware#23",sslContext=s)
content=si.content

#Parameters required are hardcoded below, please do change as per your environment.

vcha_network_name="VCHA" #port group name, I am using standard switch.
vcha_dc_name="IndiaDC"    #Datacenter name 
vcha_subnet_mask="255.255.255.0"  #Subnect mask for vCenter HA/Private network
active_vcha_ip="192.168.0.1"    # Active node vCenter HA IP
passive_vcha_ip="192.168.0.2"   # Passive node vCenter HA IP
witness_vcha_ip="192.168.0.3"   # Witness node vCenter HA IP
active_vcha_vm_name="vThinkBVM-VC1" #Active node/VC VM name
active_vc_username="Administrator@vsphere.local"  #Active VC username
active_vc_password="VMware#23"  #Active VC password
active_vc_url="https://10.61.34.35"  #Active VC public IP
active_vc_thumbprint="55:A9:C5:7E:0C:CD:46:26:D3:5C:C2:92:B7:0F:A7:91:E5:CD:0D:5D" #Active VC thumbprint
passive_vc_datastore="SharedVMFS-1"  #Passive node datastore
witness_vc_datastore="SharedVMFS-2"  #Witness node datastore

vcha=si.content.failoverClusterConfigurator #Getting managed object responsible for vCenter HA deployment

# Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
        obj = None
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for c in container.view:
                if name:
                        if c.name == name:
                                obj = c
                                break
                        else:
                                obj = None                      
        return obj


vcha_network=get_obj(content,[vim.Network],vcha_network_name) 
vcha_dc=get_obj(content,[vim.Datacenter],vcha_dc_name)

#I would highly recommend  to read vSphere API reference for "failoverClusterConfigurator", this will help to understand below specs.
 
deployment_spec=vim.vcha.FailoverClusterConfigurator.VchaClusterDeploymentSpec()

#Active node related data/parameter population
active_nw_config_spec=vim.vcha.FailoverClusterConfigurator.ClusterNetworkConfigSpec()
active_nw_config_spec.networkPortGroup=vcha_network
active_ipSettings=vim.vm.customization.IPSettings()
active_ipSettings.subnetMask = vcha_subnet_mask
active_ip_spec=vim.vm.customization.FixedIp()
active_ip_spec.ipAddress= active_vcha_ip
active_ipSettings.ip=active_ip_spec
active_nw_config_spec.ipSettings=active_ipSettings
deployment_spec.activeVcNetworkConfig=active_nw_config_spec

#Active node service locator 
active_vc_spec=vim.vcha.FailoverClusterConfigurator.SourceNodeSpec()
active_vc_vm=get_obj(content,[vim.VirtualMachine],active_vcha_vm_name)
active_vc_spec.activeVc=active_vc_vm
service_locator=vim.ServiceLocator()
cred=vim.ServiceLocator.NamePassword()
cred.username=active_vc_username
cred.password=active_vc_password
service_locator.credential=cred
service_locator.instanceUuid=si.content.about.instanceUuid
service_locator.url=active_vc_url  #Source active VC
service_locator.sslThumbprint=active_vc_thumprint
active_vc_spec.managementVc=service_locator
deployment_spec.activeVcSpec=active_vc_spec

#Passive node configuration spec
passive_vc_spec=vim.vcha.FailoverClusterConfigurator.PassiveNodeDeploymentSpec()
passive_ipSettings=vim.vm.customization.IPSettings()
passive_ipSettings.subnetMask = vcha_subnet_mask
passive_ip_spec=vim.vm.customization.FixedIp()
passive_ip_spec.ipAddress= passive_vcha_ip
passive_ipSettings.ip=passive_ip_spec
passive_vc_spec.ipSettings=passive_ipSettings
passive_vc_spec.folder=vcha_dc.vmFolder
passive_vc_spec.nodeName= active_vcha_vm_name+"-passive"
passive_datastore=get_obj(content,[vim.Datastore],passive_vc_datastore)
passive_vc_spec.datastore=passive_datastore
deployment_spec.passiveDeploymentSpec=passive_vc_spec

#Witness node configuration spec
witness_vc_spec=vim.vcha.FailoverClusterConfigurator.NodeDeploymentSpec()
witness_ipSettings=vim.vm.customization.IPSettings()
witness_ipSettings.subnetMask = vcha_subnet_mask
witness_ip_spec=vim.vm.customization.FixedIp()
witness_ip_spec.ipAddress= witness_vcha_ip
witness_ipSettings.ip=witness_ip_spec
witness_vc_spec.ipSettings=witness_ipSettings
witness_vc_spec.folder=vcha_dc.vmFolder
witness_vc_spec.nodeName=active_vcha_vm_name+"-witness"
witness_datastore=get_obj(content,[vim.Datastore],witness_vc_datastore)
witness_vc_spec.datastore=witness_datastore
deployment_spec.witnessDeploymentSpec=witness_vc_spec

# Calling the method we aimed to invoke by passing complete deployment spec

task= vcha.deployVcha_Task(deployment_spec)

if(task.info.state == "running"):
        print "VCHA deployment is started, it will take few minutes, please monitor web client for its completion"

As soon as we invoke deployVcha_Task() method, it deploys vCenter HA in basic mode where it also creates passive and witness node. Below is how it looks when looked into web client recent task pane.

We can see that “Deploy VCHA” task is running, it has in-turn cloned passive/witness nodes and initiated powereON operations. I waited for some time for deploy VCHA task to get completed. After completion, below is how vCenter HA VMs look from web client.

Note: You might have noticed that we need to have vCenter active node thumbprint to be passed in VCHA spec. I could get this thumbprint by using openssl command as shown here. In fact, there are some other ways to get it as well.

Now that we deployed VCHA successfully, let us take a look at how to destroy the vCenter HA using method “destroyVcha_Task()”: This method can be invoked when VCHA is disabled, isolated or its configuration is failed. In my case, I chose to disable the existing VCHA by using method “setClusterMode_Task()” , which we already explored in Part-I. Lets take a look at the code.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Destroy vCenter server HA
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.161.34.35", user="Administrator@vsphere.local", pwd="VMware#23",sslContext=s)
content=si.content

#Getting VCHA configurator managed object
vcha=si.content.failoverClusterConfigurator

#Getting VCHA cluster manager
vcha_cluster_manager=si.content.failoverClusterManager

# Setting vCenter HA to "disabled" mode.
task = vcha_cluster_manager.setClusterMode_Task("disabled")
while(task.info.state != "success"):
        continue

#Getting VCHA cluster mode
VCHA_mode=vcha_cluster_manager.getClusterMode()
if (VCHA_mode == "disabled"):
        vcha.destroyVcha_Task() #Destroing it
else:
        print "VCHA must be in disabled mode before destrying it"

When I executed above script, I could see from web client that vCenter HA mode got set to disabled followed by vCenter HA was getting destroyed. Please take a look at below screenshot to confirm it

I waited for few seconds & I could see, vCenter HA is destroyed as shown in below screenshot.

Note that once we destroy VCHA using above method, active node will continue to work fine & serve client requests. This API just destroys the VCHA configuration. It does not delete passive & witness node by itself. To delete passive & witness node, you will have to use regular VM destroy API (This option is provided in UI as part of vcenter HA destroy UI workflow)

Moving on to other methods exposed by managed object “FailoverClusterConfigurator”, it seems these methods (listed below) are directly suited when VCHA is deployed using advanced option

configureVcha_Task(),
createPassiveNode_Task(),
createWitnessNode_Task()

Since I had deployed vCenter HA using basic option, I will talk about above methods in my future post or will add samples on my github repo. On the other side, you can refer all the scripts discussed in part-1,2 & easily code around those as well. If you ask me, it would be really good exercise to understand vCenter HA APIs.

Notes:
1. All above scripts are available on my github repository as well. Even, I have a plan to write one nice python module for the same.
2. Note that, for the sake of simplicity, I have hard-coded some values & disabled certificate validation, hence please do appropriate changes as per your environment.
2. I highly recommend you to read Part 1
3. Finally, if you haven’t yet set up “pyVmomi” environment, refer my blog post