Tag Archives: pyvmomi

How to manage vCenter HA using vSphere python SDK pyVmomi? : Part 1

Recently I was undergoing “vSphere 6.5: Optimize and Scale” course and I was excited to see a dedicated module on “vCenter Server High Availability and Performance”. Without a doubt, vCenter HA was one of the highlights of vSphere 6.5 release. If you quickly want to understand vCenter HA, I highly recommend you to watch innovative white board style presentation by our own Adam Eckerle on VMware’s official youtube channel here

After completing the vCenter HA course lab, when I looked into vSphere 6.5 SOAP API reference, I could see there are 2 managed objects exposed as part of vCenter Server High Availability as follows.

1. FailoverClusterConfigurator
2. FailoverClusterManager

In part 1, I am going to demonstrate how can we manage vCenter HA by using 2nd managed object listed above i.e. FailoverClusterManager. This managed object provides operations to manage a vCenter High Availability Cluster (VCHA Cluster) and it assumes that vCenter HA is already deployed. Accordingly, in my case, basic vCenter HA was already deployed. Let’s now see how can we invoke methods exposed by this managed object and fetch/set some cool properties using pyVmomi. If you still haven’t set pvVmomi environment, please do configure it using my earlier post “getting started with pyVmomi”. Here we go.

If you look into API reference for managed object “FailoverClusterManager”, it has exposed total 4 methods i.e. getClusterMode(), GetVchaClusterHealth(), initiateFailover_Task() & setClusterMode_Task(). Let’s go one by one.

1. getClusterMode(): This method gets us the current mode that vCenter HA cluster is in: vCenter HA can be in one of the these 3 modes i.e. Enabled, Disabled or Maintenance mode. Let us look at what is the mode my vCenter HA cluster is in.


from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Script to get vCenter Server High Availability (VCHA) mode
# Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
c= SmartConnect(host="10.192.45.10", user="Administrator@vsphere.local", pwd="VMware!1",sslContext=s)
vcha=c.content.failoverClusterManager

VCHA_mode=vcha.getClusterMode()
print "VCHA Cluster mode::", VCHA_mode

Line 11 to 14: We have got “FailoverClusterManager” object, invoked getClusterMode() using it and finally printed the VCHA mode.

Output:
vmware@localhost:~$ python getVCHA_cluster_mode.py
VCHA Cluster mode:: enabled

Same is the VCHA cluster mode shown on below vSphere web client screenshot, is it not pretty simple to call this method?

2. GetVchaClusterHealth(): This method gives us overall health of vCenter HA.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Script to Get vCenter server HA health information

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
c= SmartConnect(host="10.192.20.30", user="Administrator@vsphere.local", pwd="VMW!23A",sslContext=s)

vcha = c.content.failoverClusterManager

VchaClusterHealth = vcha.GetVchaClusterHealth()

vcha_health_Messages = VchaClusterHealth.healthMessages
print "VCHA Health messages::"
for health_data in vcha_health_Messages:
        print health_data.message

print "\nAdditional Information::",VchaClusterHealth.additionalInformation

vcha_runtime_info = VchaClusterHealth.runtimeInfo
print "\nVCHA Cluster Mode::",vcha_runtime_info.clusterMode
print "\nVCHA Cluster State::",vcha_runtime_info.clusterState

vcha_node_info = vcha_runtime_info.nodeInfo

print "\nVCHA Node information:"
for node in vcha_node_info:
        print node.nodeRole+":"+node.nodeIp+":"+node.nodeState

Line 13: We have invoked method “GetVchaClusterHealth()” & it returns VchaClusterHealth object.
Line 15 to 18: We are fetching the property “healthMessages” offered by object returned above. Since “healthMessages” are multiple (array), we iterated and printed each one of them.
Line 20 to 30: We fetched remaining properties of the object ” VchaClusterHealth”, such as vcha mode, vcha state & vcha nodeinfo etc. and printed them one by one.

Output:
vmware@localhost:~$ python get_VCHA_health.py
VCHA Health messages::
PostgreSQL replication mode is Synchronous.
Appliance configuration is in sync.
Appliance state is in sync.
Appliance sqlite db is in sync.

Additional Information:: (vmodl.LocalizableMessage) []

VCHA Cluster Mode:: enabled

VCHA Cluster State:: healthy

VCHA Node information:
active:192.168.111.151:up
passive:192.168.111.152:up
witness:192.168.111.153:up

Note: Additional Information property returns “empty” array since VCHA cluster is in healthy mode. If cluster is not in healthy mode, it will provide additional info on what is causing the issue etc.

If you see below web client screenshot, it is matching perfectly with above output. For IPs, you can refer the 1st screenshot already posted above.

3.initiateFailover_Task(): By invoking this method, user can initiate a fail-over from active node to passive node.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

#Script to get initiate vCenter Server High Availability failover

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
c= SmartConnect(host="10.160.20.40", user="Administrator@vsphere.local", pwd="VMW!23A",sslContext=s)

vcha=c.content.failoverClusterManager
task = vcha.initiateFailover_Task(True)

while(task.info.state != "success"):
        continue
print "Initiate Failover task is completed"

Line 12 to 16: We invoked method “initiateFailover_Task(True)” and waited for task to complete. Please take a note that we are passing boolean “True” into this method, which indicates that this method will wait till on-going active-passive state replication to finish and finally it initiates the failover, other wise it will force the failover, which can lead to data loss.

As soon as I executed above script, I could see, as expected web client session was down and when I logged out of web client, I was happy to see below message on web client.

If you want to know how to initiate failover directly from web client, please take a look at below screenshot.

I waited for some more time and I could see web client was up & running again. Now I thought to check overall VCHA health to see whether VCHA is in healthy condition and active/passive nodes are actually exchanged or not. Accordingly I executed the script #2, which was written for getting VCHA health using GetVchaClusterHealth() and below was the output.

vmware@localhost:~$ python get_VCHA_health.py
VCHA Health messages::
PostgreSQL replication mode is Synchronous.
Appliance configuration is in sync.
Appliance state is in sync.
Appliance sqlite db is in sync.

Additional Information:: (vmodl.LocalizableMessage) []

VCHA Cluster Mode:: enabled

VCHA Cluster State:: healthy

VCHA Node information:
passive:192.168.111.151:up
active:192.168.111.152:up
witness:192.168.111.153:up

As expected, VCHA was in enabled mode and healthy state, even active & passive node IPs were exchanged. Same was confirmed from web client as follows.

4.setClusterMode_Task(): Using this method user can change VCHA mode. Since my current mode is “enabled” & state as “healthy”, I can either set mode to “disabled” or “maintenance”. I decided to change VCHA cluster mode to “maintenance”. In maintenance mode, VCHA active/passive state replication is enabled but automatic failover is not allowed.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Script to set vCenter Server High Availability mode
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
c= SmartConnect(host="10.192.1.2", user="Administrator@vsphere.local", pwd="VMW!23A",sslContext=s)

vcha=c.content.failoverClusterManager

task = vcha.setClusterMode_Task("maintenance")

while(task.info.state != "success"):
        continue
print "VCHA mode is set to ::", vcha.getClusterMode()

Line 11 to 15: invoked “setClusterMode_Task()” method, waited till it completes and finally checked cluster mode by calling “getClusterMode()” method.

Output:
vmware@localhost:~$ python setVCHA_cluster_mode.py
VCHA mode is set to :: maintenance

Then immediately I looked into web client and I see VCHA mode was changed to “maintenance”as follows.

That is all, How cool is that! I really enjoyed writing this post and I hope you will enjoy as well. Update: I recently published Part-2 on “FailoverClusterConfigurator” managed object as well.

Further learning on vCenter Server HA

1. All above scripts are available on my github repository as well. I have a plan to write one nice python module for the same. Note that for the sake of simplicity I have hard-coded some values & disabled certificate validation, please do appropriate changes as per your environment.
2. VCHA walkthroughs
3. If you prefer PowerCLI to play around VCHA instead of pyVmomi, please take a look at nice VCHA PowerCLI module written by none other than “William Lam
4. VCHA performance white paper
5. Finally, if you want to start learning “pyVmomi”, refer my blog post

pyVmomi tutorial : How to get all the core vCenter server inventory objects and play around?

Recently I posted about “how did I get started with pyvmomi?”. I continued exploring this further and I thought to share my learning so far. After setting up the pyvmomi environment, first thing I wanted to learn was to get hold of all core vCenter inventory objects so that I can play around each of them. vCenter core inventory includes objects such as Datacenters, Clusters, Virtual Machines, Hosts, Datastores & Networks etc.
As an initial step, I was looking for a way to get below objects from VC inventory
-All Virtual Machines
-All Clusters
-All Hosts
-All Datacenters

There could be multiple ways to get above objects but below is the way I could achieve this.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Get all the Vms from vCenter server inventory and print its name
# Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=si.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
getAllVms=get_all_objs(content, [vim.VirtualMachine])

#Iterating each vm object and printing its name
for vm in getAllVms:
        print vm.name

Output:
vmware@localhost:~$ python getAllVms.py
Database-server
NTP-Bangalore
Web-server
NTP-Pune
vmware@localhost:~$

You could see above, script listed all the VMs available in inventory. Let us look at another example, here we will get all the clusters from vCenter Inventory.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

#Get all the Clusters from vCenter invetory and printing its name
#Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=c.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
clusters=get_all_objs(content, [vim.ClusterComputeResource])

#Iterating each cluster object and printing its name
for cluster in clusters:
        print (cluster.name)

Output:
vmware@localhost:~$ python getClusters.py
DRSCluster-USA
DRSCluster-Europe
DRSCluster-India
vmware@localhost:~$

If you notice, only important change from getAllVms and getClusters script is line #22, is not it cool? So now we were able to get all the VMs as well as all the clusters. Similarly we can get other core inventory objects such as datacenter, hosts, datastores, networks etc. Below is how line #22 will look for other vCenter/ESXi objects.

# For ESXi host
hosts=get_all_objs(content, [vim.HostSystem])

#For datacenters
dcs = get_all_objs(content, [vim.Datacenter])

#For datastores 
datastores = get_all_objs(content, [vim.Datastore])

By now we know how to get hold of vSphere inventory objects, let’s do something meaningful with these objects. I chose to call a method on one of clusters that we retrieved above. What is that method?
vCenter 6.0 has introduced one of interesting APIs on vSphere DRS cluster rules. i.e. findRulesForVM(). This method is called on individual cluster and it returns set of DRS VM-VM affinity rules (enabled or disabled) those are associated with given/input VM. Since this method returns VM-VM affinity rules, I first created some VM-VM rules on one of my clusters as follows.

Cluster with DRS enabled
DRS VM-VM rules

Now that DRS rules are created, let us pass VM name as parameter to “findRulesForVM()” API and call this API on that cluster.

Updated code is available on VMware sample exchange portal here

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Get DRS VM-VM rules associated with a VM.

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$vabc325pqr",sslContext=s)
content=c.content

def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

# Scanning a input VM inside inventory using special  python construct i.e. List comprehension
# It will get all the Vms and check whether input VM is available inside inventory or not, finally it returns list with matching condition

vmToScan = [vm for vm in get_all_objs(content,[vim.VirtualMachine]) if "NTP-Bangalore" == vm.name]

# Scanning a input cluster inside invetory the way we did for VM above. here also we used list comprehension.

cluster = [cluster for cluster in get_all_objs(content,[vim.ClusterComputeResource]) if "DRSCluster-India" == cluster.name]

# Now we can call the method on input cluster by passing input VM as parameter, it returns array of rule objects associated with input VM.
ClusterRuleInfo=cluster[0].FindRulesForVm(vmToScan[0])

# Now iterate through rule objects and print the rule name

for rule in ClusterRuleInfo:
        print rule.name

Output:
vmware@localhost:~$ python findRules.py
Anti-Affine-NTP-Servers
Affine-Web-DB-Servers

Here is how vSphere web client “recent task” looks, you could see “Find Rules for vms” task.

Notes:
-For the sake of simplicity, I have hard-coded some values and simplified the code, please make changes as per your requirement.
-I have added required documentation inside above code itself.
-This code follows mostly python 2.7.x version, please make minor changes to work with python 3.x

If you need any help, please let me know and yes, please stay tuned for my upcoming blog posts on pyVmomi.