Tag Archives: vSphere APIs

Configure and Audit Latency Sensitivity enabled VMs on vSphere using vSphere API

Recently I had an opportunity to work on “Latency sensitivity feature” which was introduced in vSphere 5.5 release. This feature is one of the reasons why mission critical VMs can be run on vSphere infrastructure or vSphere enabled clouds. In this blog post, my focus is on sharing how we can configure and audit latency sensitivity enabled VMs using vSphere Java SDK. Before looking into Java SDK scripts, lets understand what is this feature in brief. Latency sensitivity feature can be enabled per VM level (From vSphere web client >> Manage >> VM options >> Latency sensitivity set to high), this feature enables vSphere admin to allocate exclusive access to physical resources for the VM, reserving 100% vCPU time ensures that exclusive PCPU access to the VM. Once vCPU time is reserved (in MHz), this features requires to reserve 100% of the VM memory. 100% memory reservation ensures memory availability for the VM, in other words, memory will not be reclaimed (in case of memory contention) from the VM as it is 100% reserved. Overall, this feature enables vSphere Admin to bypass/tune the virtualization layer and have exclusive access to the hosts physical resources. For more details on this feature I highly recommend to read this official whitepaper

So now we understood that, in order to properly configure latency sensitive VMs, admin not only needs to enable this feature but also reserve 100% of vCPU time in Mhz and 100% memory. Setting memory reservation equal to VMs configured memory is straight-forward but what about allocating vCPU time (in Mhz)? How to set proper vCPU reservation? Lets consider one example: Lets say ESXi host’s PCPU speed is 2.5 Ghz (2500 Mhz) per core. Now if your VM has 2 vCPU configured, vCPU reservation must be 2*2500=5000 Mhz i.e. Number of vCPUs*PCPU speed per core. Similarly we need to follow for any number of vCPU configured per VM. There is one more interesting point i.e. vSphere DRS interop with Latency sensitivity enabled VMs. When such VM is there inside vSphere DRS enabled cluster, one VM-Host soft rule gets created internally(not visible from Web client) with such VM and the host on which this VM is currently residing. As rule is soft, if needed (in case of imbalanced cluster) DRS can violate this rule and migrate this VM to another host. As soon as DRS migrates this VM to another host, earlier rule gets deleted automatically and new rule gets created with new host. You may ask why is this rule gets created internally. Purpose is to avoid vMotion (by DRS) of such latency sensitive VM as much as possible as vMotion can lead to performance impact though its low. (Pass-through with latency sensitive VMs can be leveraged as well, this is explained very well in above pointed white paper)

Now lets take a look at Java SDK scripts. There are 2 Java SDK scripts, one is to configure LS VM and another is for auditing all the LS enabled VMs.

1. Script to configure latency sensitivity feature on the VM

Same script can be found on VMware’s sample exchange or on my git hub account here


//:: # Author: Vikas Shitole
//:: # Website: www.vThinkBeyondVM.com
//:: # Product/Feature: vCenter Server/Latency Sensitivity of the VM(Exclusive pCPU-vCPU affinity)
//:: # Description: Script to configure latency sensitivity feature on the VM and getting list of VMs where LS is configured.
//:# How to run this sample: http://vthinkbeyondvm.com/getting-started-with-yavi-java-opensource-java-sdk-for-vmware-vsphere-step-by-step-guide-for-beginners/
//:# Reference: http://www.vmware.com/files/pdf/techpaper/latency-sensitive-perf-vsphere55.pdf

package com.vmware.yavijava;

import java.net.MalformedURLException;
import java.net.URL;
import java.rmi.RemoteException;
import com.vmware.vim25.InvalidProperty;
import com.vmware.vim25.LatencySensitivity;
import com.vmware.vim25.LatencySensitivitySensitivityLevel;
import com.vmware.vim25.ResourceAllocationInfo;
import com.vmware.vim25.RuntimeFault;
import com.vmware.vim25.TaskInfoState;
import com.vmware.vim25.VirtualHardware;
import com.vmware.vim25.VirtualMachineConfigSpec;
import com.vmware.vim25.mo.Folder;
import com.vmware.vim25.mo.HostSystem;
import com.vmware.vim25.mo.InventoryNavigator;
import com.vmware.vim25.mo.ManagedEntity;
import com.vmware.vim25.mo.ServiceInstance;
import com.vmware.vim25.mo.Task;
import com.vmware.vim25.mo.VirtualMachine;
import com.vmware.vim25.mo.util.MorUtil;

public class ConfigLatencySensitivity {

public static void main(String[] args) throws InvalidProperty,
RuntimeFault, RemoteException, MalformedURLException {

if(args.length!=5)
{
System.out.println("Usage: Java ConfigLatencySensitivity vCurl username password hostname/IP VMName");
System.exit(-1);
}

URL url = null;
try
{
url = new URL(args[0]);
} catch ( MalformedURLException urlE)
{
System.out.println("The URL provided is NOT valid. Please check it.");
System.exit(-1);
}

ServiceInstance si = new ServiceInstance(new URL(args[0]), args[1],
args[2], true); // Pass 3 argument as vCenterIP/username/password

String VMHost=args[3]; //Host on which VM is resided
String VMToBeConfigured=args[4]; //VM name to be configured.

Folder rootFolder = si.getRootFolder();
VirtualMachine vm1 = (VirtualMachine) new InventoryNavigator(rootFolder)
.searchManagedEntity("VirtualMachine", VMToBeConfigured);

HostSystem host = (HostSystem) new InventoryNavigator(rootFolder)
.searchManagedEntity("HostSystem", VMHost);

//Check how many Mhz CPU can be reserved per vCPU of the VM
int cpuMhz=host.getSummary().getHardware().getCpuMhz();
System.out.println("cpuMHz that can be reserved per vCPU::"+cpuMhz);

//Get RAM and vCPU configured to VM while creating that VM
VirtualHardware vHw=vm1.getConfig().getHardware();
int vmMem=vHw.getMemoryMB();
System.out.println("RAM of the VM " +vm1.getName()+" ::"+vmMem);
int vCpu=vHw.getNumCPU();
System.out.println("Number of vCPUs configured on VM "+vm1.getName()+ " are::"+ vCpu);

VirtualMachineConfigSpec vmSpec=new VirtualMachineConfigSpec();

//Set the latency sensitive level flag to "high"
LatencySensitivity ls=new LatencySensitivity();
ls.setLevel(LatencySensitivitySensitivityLevel.high);
vmSpec.setLatencySensitivity(ls);

// It is highly recommended to reserve the CPU in Mhz equal to Multiples of vCPU
ResourceAllocationInfo cpuAlloc=new ResourceAllocationInfo();
cpuAlloc.setReservation((long) (vCpu*cpuMhz));
vmSpec.setCpuAllocation(cpuAlloc);

//It is highly recommended to reserve the memory equal to RAM of the VM
ResourceAllocationInfo memAlloc=new ResourceAllocationInfo();
memAlloc.setReservation((long) vmMem);
vmSpec.setMemoryAllocation(memAlloc);

//Reconfigure the VM and check reconfigure task status
Task task=vm1.reconfigVM_Task(vmSpec);
System.out.println("Reconfigure Task started ......");

//Wait till task is either queued or running, that will help us to verify whether task is successful or not
while(task.getTaskInfo().getState().equals(TaskInfoState.queued)||task.getTaskInfo().getState().equals(TaskInfoState.running) ){
System.out.println(task.getTaskInfo().getState());
}
if(task.getTaskInfo().getState().equals(TaskInfoState.success))
{
System.out.println("Latency sensitive feature is enabled on the VM "+vm1.getName()+" successfully");
}else{

System.out.println("Latency sensitive feature is NOT enabled on the VM "+vm1.getName()+" successfully");
}

//List of VMs with latency sensitivity enabled will be printed for reference.
System.out.println("==============================================================");
System.out.println("List of VMs where Latency sensitivity feature is enabled");
System.out.println("==============================================================");
ManagedEntity[] vms = new InventoryNavigator(rootFolder).searchManagedEntities("VirtualMachine");
for(ManagedEntity vm:vms){
VirtualMachine vmMo = (VirtualMachine) MorUtil
.createExactManagedEntity(si.getServerConnection(), vm.getMOR());
if(vmMo.getConfig().getLatencySensitivity().getLevel().equals(LatencySensitivitySensitivityLevel.high)){

System.out.println(vmMo.getName());
}
}

si.getServerConnection().logout();
}

}

2. Script to audit Latency sensitivity feature configuration on all the VMs in a vCenter Server.

Same script can be found on VMware’s sample exchange or on my git hub repo here

//:: # Author: Vikas Shitole
//:: # Website: www.vThinkBeyondVM.com
//:: # Product/Feature: vCenter Server/Latency Sensitivity of the VM(Exclusive pCPU-vCPU affinity)
//:: # Description: Script to audit Latency sensitivity feature configuration on all the VMs in a vCenter Server.
//:# How to run this sample: http://vthinkbeyondvm.com/getting-started-with-yavi-java-opensource-java-sdk-for-vmware-vsphere-step-by-step-guide-for-beginners/
//:# Reference: http://www.vmware.com/files/pdf/techpaper/latency-sensitive-perf-vsphere55.pdf

package com.vmware.yavijava;

import java.util.HashMap;
import java.util.Map;
import java.net.MalformedURLException;
import java.net.URL;
import java.rmi.RemoteException;
import com.vmware.vim25.InvalidProperty;
import com.vmware.vim25.LatencySensitivitySensitivityLevel;
import com.vmware.vim25.RuntimeFault;
import com.vmware.vim25.VirtualHardware;
import com.vmware.vim25.VirtualMachineConfigInfo;
import com.vmware.vim25.mo.Folder;
import com.vmware.vim25.mo.HostSystem;
import com.vmware.vim25.mo.InventoryNavigator;
import com.vmware.vim25.mo.ManagedEntity;
import com.vmware.vim25.mo.ServiceInstance;
import com.vmware.vim25.mo.VirtualMachine;
import com.vmware.vim25.mo.util.MorUtil;

public class AuditLatencySensitivityConfig {

public static void main(String[] args) throws InvalidProperty,
RuntimeFault, RemoteException, MalformedURLException {

if(args.length!=3)
{
System.out.println("Usage: Java AuditLatencySensitivityConfig vCurl username password ");
System.exit(-1);
}

URL url = null;
try
{
url = new URL(args[0]);
} catch ( MalformedURLException urlE)
{
System.out.println("The URL provided is NOT valid. Please check it.");
System.exit(-1);
}
String username = args[1]; //vCenter username
String password = args[2]; //vCenter password

// Initialize the system, set up web services

ServiceInstance si = new ServiceInstance(url, username,
password, true); // Pass 3 argument as vCenter URL/username/password

Folder rootFolder = si.getRootFolder();
System.out.println("===========================================================================================");
System.out.println("Auditing configuration of Latency Sensitivity Enabled VMs");
System.out.println("Audit criteria:");

System.out.println("1. VM with LS level set to high should have CPU reservation in multiples of vCPU configured");
System.out.println("2. VM with LS level set to high should have memory reservation equal to VM RAM");
System.out.println("===========================================================================================");

//Maps to store VMs with LS configured and mis-configured
Map lsConfiguredVms=new HashMap();
Map
lsMisConfiguredVms=new HashMap();

//Getting all the hosts available in vCenter Server inventory
ManagedEntity[] hosts = new InventoryNavigator(rootFolder).searchManagedEntities("HostSystem");

//Iterating through 1 host at a time as follows
for(ManagedEntity host1 :hosts){

HostSystem host = (HostSystem) MorUtil
.createExactManagedEntity(si.getServerConnection(), host1.getMOR());

//Check how many Mhz CPU can be reserved per vCPU wrt specified host
int cpuMhz=host.getSummary().getHardware().getCpuMhz();

//Getting all the VMs available on the host
ManagedEntity[] vms = new InventoryNavigator(host).searchManagedEntities("VirtualMachine");
if(vms!=null){
//Iterating through each and very VMs available on perticular host
for(ManagedEntity vm:vms){
VirtualMachine vmMo = (VirtualMachine) MorUtil
.createExactManagedEntity(si.getServerConnection(), vm.getMOR());

//Check whether latency sensitivity property set on the hosts or not
if(vmMo.getConfig().getLatencySensitivity().getLevel().equals(LatencySensitivitySensitivityLevel.high)){
VirtualHardware vHw=vmMo.getConfig().getHardware();
int vmMem=vHw.getMemoryMB(); //RAM of the VM
int vCpu=vHw.getNumCPU(); //vCPUs configured to the VM
VirtualMachineConfigInfo vmConfigInfo=vmMo.getConfig();
long cpuReservation=vmConfigInfo.getCpuAllocation().getReservation(); //CPU reservation on the VM
long memReservation=vmConfigInfo.getMemoryAllocation().getReservation(); //Memory reservation on the VM

//Compare mem/cpu reservation wrt configured memory/vCPUs on the host
if((cpuReservation==(vCpu*cpuMhz))&&(memReservation==vmMem)){
lsConfiguredVms.put(vmMo.getName(),host.getName());
}else{
lsMisConfiguredVms.put(vmMo.getName(),host.getName());
}
}
}
}else{
System.out.println("NO VMs available on the specified host");
}
}
System.out.println("VM list where Latency sensitivity is configured properly");
System.out.println("-------------------------------------------------------------");
for (Map.Entry< String , String> lsConfiguredVm : lsConfiguredVms.entrySet()) {
System.out.println("[" + lsConfiguredVm.getKey() + "::" + lsConfiguredVm.getValue()
+ "]");
}
System.out.println("\n VM list where Latency sensitivity is NOT configured properly");
System.out.println("-------------------------------------------------------------");
for (Map.Entry< String , String> lsMisConfiguredVm : lsMisConfiguredVms.entrySet()) {
System.out.println("[" + lsMisConfiguredVm.getKey() + "::" + lsMisConfiguredVm.getValue()
+ "]");
}
si.getServerConnection().logout();
}
}

Let me know if you need any help on this. If you have still not setup your YAVI JAVA Eclipse environment:Getting started tutorial Important tutorials to start with: Part I & Part II

DRS rules PART I: How to create VM-Host DRS affinity rule using vSphere API

On high level there are 2 types of DRS rules, one is VM-VM affinity rules and other is VM-Host affinity rules. Today in Part I, I will brief on what are the VM-Host affinity rules DRS has & what are the use cases where VM-Host rules can be used. However more focus would be on how to create VM-Host rules on ESXi cluster using vSphere API. In general, VM-Host rules will allow us to run particular group of VMs on specific group of ESXi hosts or away from specific group of ESXi hosts.

What are the DRS VM-Host rules we can create on ESXi host cluster.
1. VM-Host must affinity rule : It is mandatory rule where specific group of VMs must run on specific group of hosts.
2. VM-Host must anti-affinity rule : It is mandatory rule where specific group of VMs must not run on specific group of hosts.
3. VM-Host should affinity rule : It is soft rule where specific group of VMs should run on specific group of hosts. DRS or user can violate this rule whenever required. But DRS will make best effort to correct the violation in the next DRS invocation.
4. VM-Host should anti-affinity rule : It is soft rule where specific group of VMs should not run on specific group of hosts. DRS or user can violate this rule whenever required. But DRS will make best effort to correct the violation in the next DRS invocation.

What are the use cases where we can use these rules
1. There are some products their licenses are based on ESXi host CPUs, it is mean that you need to apply licenses on these ESXi host to run the VMs. Classic example is Oracle licensing for databases. In this case, we can configure VM-Host must affinity rule so that oracle DB VMs will only run on set of hosts with oracle license.
2. VM host anti-affinity rule can be used to increase the availability of the VM. i.e. Running some critical VMs on set of hosts with different power supplies than rest of the host in the cluster so that even when there is power failure, some critical VMs can be still running on set of host with different power supplies.
I strongly suggest you to read my blog post is vSphere HA aware of DRS rules in order to understand impact of the DRS rules on vSphere HA.

Now it is time to look into creating these rules using vSphere APIs. Before jumping on developing code, It would be great if you get familiar with ClusterConfigSpecEx data object required for creating VM-Host rules. Specifically focus on data object properties high-lighted in below screenshot.
RuleSpec_GroupSpec

Below code is for creating VM-Host must affinity rule, I will explain how to leverage this code to create other 3 rules. It is really very simple.

package com.vmware.vijava;
import java.net.URL;
import com.vmware.vim25.ArrayUpdateOperation;
import com.vmware.vim25.ClusterConfigSpecEx;
import com.vmware.vim25.ClusterGroupInfo;
import com.vmware.vim25.ClusterGroupSpec;
import com.vmware.vim25.ClusterHostGroup;
import com.vmware.vim25.ClusterRuleSpec;
import com.vmware.vim25.ClusterVmGroup;
import com.vmware.vim25.ClusterVmHostRuleInfo;
import com.vmware.vim25.ManagedObjectReference;
import com.vmware.vim25.mo.ClusterComputeResource;
import com.vmware.vim25.mo.Folder;
import com.vmware.vim25.mo.HostSystem;
import com.vmware.vim25.mo.InventoryNavigator;
import com.vmware.vim25.mo.ServiceInstance;
import com.vmware.vim25.mo.VirtualMachine;
import com.vmware.vim25.mo.util.MorUtil;

public class VMHOSTrule {

public static void main(String[] args) throws Exception {
if (args.length != 3) {
System.out.println("Usage: java SearchDatastore [url] "
+ "[username] [password]");
return;
}

ServiceInstance si = new ServiceInstance(new URL(args[0]), args[1],
args[2], true);
/*
* you need to pass 3 parameters 1. https://x.y.z.r/sdk 2. username 3.
* password. Plz connect to vCenter Server
*/
String vmGroupName = "vmGroup_1";
String hostGroupName = "HostGroup_1";
Folder rootFolder = si.getRootFolder();

ClusterComputeResource clu = null;

clu = (ClusterComputeResource) new InventoryNavigator(rootFolder)
.searchManagedEntity("ClusterComputeResource", "India_Cluster");
ManagedObjectReference ClusterMor = clu.getMOR();

HostSystem host1 = (HostSystem) new InventoryNavigator(rootFolder)
.searchManagedEntity("HostSystem", "10.10.1.1");
HostSystem host2 = (HostSystem) new InventoryNavigator(rootFolder)
.searchManagedEntity("HostSystem", "10.10.1.2");
ManagedObjectReference hostMor1 = host1.getMOR();
ManagedObjectReference hostMor2 = host2.getMOR();
VirtualMachine vm1 = (VirtualMachine) new InventoryNavigator(rootFolder)
.searchManagedEntity("VirtualMachine", "CentOS6_x64_2GB_1");
VirtualMachine vm2 = (VirtualMachine) new InventoryNavigator(rootFolder)
.searchManagedEntity("VirtualMachine", "CentOS6_x64_2GB_2");
ManagedObjectReference vmMor1 = vm1.getMOR();
ManagedObjectReference vmMor2 = vm2.getMOR();
ClusterComputeResource ccr = (ClusterComputeResource) MorUtil
.createExactManagedEntity(si.getServerConnection(), ClusterMor);
ManagedObjectReference[] vmMors = new ManagedObjectReference[] {
vmMor1, vmMor2 };
ManagedObjectReference[] hostMors = new ManagedObjectReference[] {
hostMor1, hostMor2 };

ClusterGroupInfo vmGroup = new ClusterVmGroup();
((ClusterVmGroup) vmGroup).setVm(vmMors);
vmGroup.setUserCreated(true);
vmGroup.setName(vmGroupName);

ClusterGroupInfo hostGroup = new ClusterHostGroup();
((ClusterHostGroup) hostGroup).setHost(hostMors);
hostGroup.setUserCreated(true);
hostGroup.setName(hostGroupName);

ClusterVmHostRuleInfo vmHostAffRule = new ClusterVmHostRuleInfo();
vmHostAffRule.setEnabled(new Boolean(true));
vmHostAffRule.setName("VMHOSTAffinityRule");
vmHostAffRule.setAffineHostGroupName("HostGrp_1");
vmHostAffRule.setVmGroupName("VMGrp_1");
vmHostAffRule.setMandatory(true);
ClusterGroupSpec groupSpec[] = new ClusterGroupSpec[2];
groupSpec[0] = new ClusterGroupSpec();
groupSpec[0].setInfo(vmGroup);
groupSpec[0].setOperation(ArrayUpdateOperation.add);
groupSpec[0].setRemoveKey(null);
groupSpec[1] = new ClusterGroupSpec();
groupSpec[1].setInfo(hostGroup);
groupSpec[1].setOperation(ArrayUpdateOperation.add);
groupSpec[1].setRemoveKey(null);
/* RulesSpec for the rule populated here */
ClusterRuleSpec ruleSpec[] = new ClusterRuleSpec[1];

ruleSpec[0] = new ClusterRuleSpec();
ruleSpec[0].setInfo(vmHostAffRule);
ruleSpec[0].setOperation(ArrayUpdateOperation.add);
ruleSpec[0].setRemoveKey(null);
ClusterConfigSpecEx ccs = new ClusterConfigSpecEx();
ccs.setRulesSpec(ruleSpec);
ccs.setGroupSpec(groupSpec);

ccr.reconfigureComputeResource_Task(ccs, true);
System.out.println("VM Host Rule created successfully");

}

}


Line 22-29: Refer my this blog post on ServiceInstance data object to understand these lines.
Line 41-54: Refer my this blog post on Inventory Navigator to understand these lines.
Line 64-72: We are populating VMGroup i.e. Set of VMs of those will be part of rule. & HostGroup i.e. set of hosts those will be part of rule. These groups will be pinned with GroupSpec in line 80-88.
Line 74-78: Creation of ClusterVmHostRuleInfo data object which helps to identify which host group and VM Group is associated with this rule as there can be several other host and VM groups associated with other rules already exists or planned to configure. Note that line 77 is responsible in deciding whether rule is affinity or anti affinity.
Line 80-88: Here we are populating GroupSpec required for creating VM & Host group on cluster based on the line 64-72.
Line 90-95: Here we are populating RuleSpec based on line 74-78.
Line 96-98: Populating ClusterConfigSpecEx data object with RuleSpec and GroupSpec which will be passed to reconfigureComputeResource_Task method which actually re-configures cluster with created VM-Host must affinity rule.

Here is the created VM-Host rule view from vSphere Client.
VM Host Rule

Noteworthy points:
1. For the sake of simplicity we have hard-coded some IPs, cluster, host & VMs names. Please do make changes if you are using this in production environment.
2. In order to create anti-affinity rule, you will just need to replace line 77 to vmHostAffRule.setAntiAffineHostGroupName(“HostGrp_1”); is it not too simple?.
3.In order to make this rule soft, just replace line 79 to vmHostAffRule.setMandatory(false); Note that default rule created is soft rule(i.e. If we do not set the property).

I hope you enjoyed this post, stay tuned for PART II on VM-VM affinity rules.

Learn more on VM-Host rules here.
If you have still not setup your Eclipse environment:Getting started tutorial