All posts by vThinkBeyondVM

About vThinkBeyondVM

Vikas Shitole is currently working at VMware as Senior Member of Technical Staff (MTS) where he primarily contribute to the VMware vCenter server team which is part of VMware’s R&D organisation. He has been vExpert since last 4 years in row (2014-2017)-title awarded by VMware for significant contribution to VMware community.

vSphere 6.7: What is per-VM EVC and How to configure & manage it using pyVmomi?

Enhanced vMotion Compatibility (EVC) is without a doubt has been one of the famous vSphere features. As you know already,  EVC is a cluster level feature, which makes it possible to have vMotion across different generations of CPU within the cluster.  vSphere 6.7 has taken EVC to the next level.  This latest release has introduced one of cool features around EVC i.e. per-VM EVC.  Recently I got an opportunity to explore this cool feature and I thought to share my learning with you. In this blog post, I will take a through below items around per-VM EVC.

  • What is per-VM EVC?
  • Notes on per-VM EVC
  • per-VM EVC UI workflow
  • Configuring per-VM EVC using pyVmomi
  • Playing around featureMask using pyVmomi
  • Disabling per-VM EVC using pyVmomi
What is per-VM EVC?

As name indicates, per -VM EVC can be enabled on individual VM. Great thing about per VM EVC is that, it not only works on VMs inside the cluster but also VMs outside of the cluster.  Unlike cluster level EVC, this feature enables vMotion across clusters, standalone hosts, cross vCenters & hybrid clouds such as VMware cloud on AWS (from on-prem vCenter). How cool is that?

Notes on per-VM EVC
  • User needs to have vSphere 6.7
  • VM hardware version must be vmx-14
  • User must powered off the VM before configuring per-VM EVC
  • If user is enabling per -VM EVC on a VM, which is inside EVC enabled cluster, EVC mode on the VM should be equal or lower than that of EVC mode configured on cluster.
  • If  user wants to enable this feature from UI, only vSphere H5 client  supports it and not the flex based client.
  • per-VM EVC works fine with vSphere DRS
  • If user clones per-VM EVC configured VM, it will retain per-VM EVC configuration on cloned VM.
  • If underlying host does not support EVC mode configured on VM, VM can not power ON
  • User can enable per-VM EVC on a VM though VM is already part of a EVC enabled cluster.
per-VM EVC UI workflow

If you ask me, it is pretty easy to configure per-VM EVC from vSphere H5 client (flex client does not support). User just needs to click on the VM >> Configure >> VMware EVC >>Edit >> Configure EVC mode of your choice (of course EVC mode supported by underlying host/Cluster). Please take a look at below H5 client screenshots.

Per VM EVC UI configuration workflow

I configured “Intel ivy-bridge” EVC mode and below is how it looks like post configuration.

per VM EVC UI post configuration
Configuring per-VM EVC using pyVmomi

vSphere 6.7 has exposed a vSphere API to configure per-VM EVC i.e. ApplyEvcModeVM_Task(). We need to pass right “featureMask” to this API in order to configure appropriate EVC mode on the VM. Note that every EVC mode defines its own set of featureMask. Ex. intel-sandybridge EVC mode will have corresponding set of featureMask, so is for intel-ivybridge and so on..

Since we need to pass “featureMask” for particular EVC mode, first important thing user needs to get hold of is right EVC mode. Once we get hold of right EVC mode, we need to get corresponding  “featureMask”.  In our case, lets configure per-VM EVC on a VM residing  on a standalone host.  As I said, first we need to find max EVC mode supported on the host. This can be easily found using the script I discussed in my blog post here (Refer section on “Getting max EVCMode key“). It can be quickly found from vSphere web client or H5 client host summary as well. In my case, my host “max EVCMode”was “intel-haswell”. It does mean that I can enable per-VM EVC with featureMask for either “intel-haswell” EVC mode or lower EVCMode such as “intel-ivybridge” , “intel-sandybridge” etc.  I chose to enable per-VM EVC on “intel-ivybridge” EVC mode.

Now that we have finalized EVC mode i.e. intel-ivybridge, we now need to get hold of corresponding “featureMask”. Please take a look at below code snippet.

si= SmartConnect(host=args.host, user=args.user, pwd=args.password, sslContext=s)
supported_evc_mode=si.capability.supportedEVCMode
for evc_mode in supported_evc_mode:
	if(evc_mode.key == "intel-ivybridge"):
		ivy_mask=evc_mode.featureMask
		break

If you take a look above snippet, it is fairly easy to get featuremask for particular EVC Mode.

putting it together

This script is available on my github-repo here

 
# Author: Vikas Shitole
# Website: www.vThinkBeyondVM.com
# Product: vCenter server/ per-VM EVC (Enhanced Compatibility Mode)
# Description: Script to get enbale/disable per-VM EVC on VM
# Reference: 
# How to setup pyVmomi environment?:
# On linux:  http://vthinkbeyondvm.com/how-did-i-get-started-with-the-vsphere-python-sdk-pyvmomi-on-ubuntu-distro/
# On windows: http://vthinkbeyondvm.com/getting-started-with-pyvmomi-on-windows-supports-vsphere-6-7/


from pyVim.connect import SmartConnect
import ssl
from pyVmomi import vim
import atexit
import sys
import argparse
import getpass
 
 
def get_args():
    """ Get arguments from CLI """
    parser = argparse.ArgumentParser(
        description='Arguments for talking to vCenter')

    parser.add_argument('-s', '--host',
                        required=True,
                        action='store',
                        help='vSpehre service to connect to')

    parser.add_argument('-o', '--port',
                        type=int,
                        default=443,
                        action='store',
                        help='Port to connect on')

    parser.add_argument('-u', '--user',
                        required=True,
                        action='store',
                        help='Username to use')

    parser.add_argument('-p', '--password',
                        required=False,
                        action='store',
                        help='Password to use')

    parser.add_argument('-v', '--vmname',
                        required=True,
                        action='store',
                        default=None,
                        help='Name of the VM to be configured per VM EVC')	

    args = parser.parse_args()

    if not args.password:
        args.password = getpass.getpass(
            prompt='Enter vCenter password:')

    return args
 
 # Below method helps us to get MOR of the object (vim type) that we passed.
def get_obj(content, vimtype, name):
 obj = None
 container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
 for c in container.view:
  if name and c.name == name:
   obj = c
   break
 container.Destroy()
 return obj
 
 
args = get_args() 
s=ssl.SSLContext(ssl.PROTOCOL_SSLv23) # For VC 6.5/6.0 s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
 
si= SmartConnect(host=args.host, user=args.user, pwd=args.password, sslContext=s)
content=si.content
vm= get_obj(content, [vim.VirtualMachine],args.vmname)

if(vm and vm.capability.perVmEvcSupported):
        print ("VM available in vCenter server and it supports perVm EVC, thats good")
else:
        print ("VM either NOT found or perVMEvc is NOT supported on the VM")
        quit()

supported_evc_mode=si.capability.supportedEVCMode
for evc_mode in supported_evc_mode:
	if(evc_mode.key == "intel-ivybridge"):
		ivy_mask=evc_mode.featureMask
		break
	 
vm.ApplyEvcModeVM_Task(ivy_mask,True)
print ("ApplyEvcModeVM_Task() API is invoked, check out your H5 client")

Line #80: This confirms whether VM found or not and it also confirms whether VM supports per-VM EVC or not.
#86-90: It is about getting right featureMask corresponding to particular EVC Mode.
#92: Finally we called the API “ApplyEvcModeVM_Task()”

How to run this script
“python perVMEVCNew.py -s 10.161.81.159 -u Administrator@vsphere.local -v DRSPerVMEVC”, user needs to enter vCenter password. Below is how it looks like.

I looked into vSphere H5 client and this is how it looks like. isn’t it cool?

Playing with featureMask

In line #86, we learned how to get hold of featureMask. is it the only way we can get featureMask and pass it to this API? Answer is NO. There are multiple ways. However, I recommend to follow the way I did on line #86 above.
Sometime you may want to configure per-VM EVC with the same featureMask as that of EVC enabled cluster.  This can be handy specially for the VMs those are outside of the EVC cluster. In such case, you can simply copy the featureMask from EVC cluster and pass it to the per-VM EVC API. Let us take look at below code snippet.

#Cluster object
cluster = get_obj(content,[vim.ClusterComputeResource], args.cluster)

if(cluster):
        print ("Cluster available in vCenter server, thats good")
else:
        print ("Cluster is NOT available in vCenter server, please enter correct name")
        quit()
		
evc_cluster_manager=cluster.EvcManager()

evc_state=evc_cluster_manager.evcState
current_evcmode_key= evc_state.currentEVCModeKey

if(current_evcmode_key):
        print ("Current EVC Mode::"+current_evcmode_key)
else:
        print ("EVC is NOT enabled on the cluster")
        quit()
features_masked = evc_state.featureMask

Above EVC API property I had already discussed in my post on cluster level EVC APIs here (Refer: section #4 : Exploring EVC Cluster state). Here is the complete script, where featureMask is copied from EVC cluster. In addition, you can also choose to copy the featureMask from already per-VM EVC configured VM but if you do not document it properly, it can lead to confusion in future, hence it is always better to follow one way consistently to avoid any issues in future.

Disabling per-VM EVC

If you ask me, it is pretty easy. We just need to invoke the same API but without any featureMask as follows


vm.ApplyEvcModeVM_Task(None,True)

Some useful resources on EVC

1. Part-1: Managing EVC using pyVmomi
2. Part 2: Managing EVC using pyVmomi
3. Tutorial on getting started pyVmomi  on linux
4. Tutorial on getting started pyVmomi on Windows

I hope you enjoyed this post, let me know if you have any questions/doubts.

New fling : DRS Entitlement Viewer: a feature walk-through

A cool new fling was just released i.e. DRS entitlement viewer. As stated on fling site

“DRS entitlement viewer gives the hierarchical view of vCenter DRS cluster inventory with entitled CPU and memory resources for each resource pool and VM in the cluster”.

Good thing about this fling is that it will be deployed as HTML5 client plugin, which enables you to enjoy its features from vSphere H5 client itself. As requirement we need to have “vCenter server Appliance 6.5 or 6.7”.  Let us take a look at how to deploy. I fyou ask me, it is pretty easy and quick.

  • Fling Deployment

Step 1: Unzip the plugin package (downloaded zip bundle from fling site) to directory “/usr/lib/vmware-vsphere-ui/plugin-packages/” . Below is how it should look like. Note the exact fling directory and its content.

Step 2: Setting advanced options “CompressDrmdumpFiles set to 0” and “DrmdumpResActions set to 1”. Below is how it looks.

You can also add these DRS cluster level advanced options using below PowerCLI code. You can extend this code to configure these options on all the DRS clusters in your vCenter server.


$Cluster = Get-Cluster "Your cluster Name"
$Cluster | New-AdvancedSetting -Name CompressDrmdumpFiles -Value 0 -Type ClusterDRS
$Cluster | New-AdvancedSetting -Name DrmdumpResActions -Value 1 -Type ClusterDRS

 

Step 3: Restart vSphere H5 client service using below commands.
service-control –stop vsphere-ui
service-control –start vsphere-ui

Above step can be automated using vCenter REST APIs as well (either using PowerCLI or any other programming language without SSH connection)

I think, even first step can be automated using “ExtensionManager() vSphere API” without making any SSH connection to vCenter server appliance. (more on this in future post)

  • Fling features

As soon as vSphere H5 client is up, we should be able to see  the new “DRS entitlements” section under the “Monitor” tab for each Cluster. When I logged into my local H5 client, I was able to see below DRS entitlement from CPU and Memory perspective.

You can see DRS entitlement for every VM is showcased. You may wonder how come CPU entitlement is 0 MHz?. The reason is that, all VMs are completely idle, hence that is absolutely expected. As a first step, you just need to click on “recompute” to get the current DRS entitlement across all VMs/ResourcePools. Note that when you actually do some RLS changes on clustered VMs/RPs and you would like to see updated DRS entitlement, you will have to either click “Run DRS” or wait till default DRS execution (every 5 min).

One of the most impressive features of this fling is having ability to do “What if” analysis. Below 3 super cool scenarios are supported.

#1. What if user modifies RLS settings across VMs or resource pools across clusters. (RLS : Reservation, Limit, Shares). To try out this, first select RLS radio button and you can choose to simulate the RLS settings for one or more VMs/RPs (there is settings icon for every VM/RP except root). Finally click on “submit” button to get an idea on change in DRS entitlement for those VMs/RPs for which RLS settings were modified. Please take a look at below couple of screenshots to understand this better.

I simulated one of VM’s memory reservation to “500 MB” and once I “submit” this change, below is how What if analysis looks like.

#2: What if all the VMs’ resource demand is at 100%. I did try this analysis and below is how it was looking like. How cool is that?

#3: You can have What if analysis for both RLS and 100% both at the same time

Note: As specified above, RLS changes are just simulations, it does not actually get updated on VMs/RPs. How cool it is to do this analysis without even updating actual RLS settings on cluster!  Also,  I would highly recommend you to read the user guide to understand all What if scenarios in detail and rest of the features.

Another impressive feature is to have option to get “PowerCLI” code snippet to actually update RLS settings once your analysis is done. Please take a look at one of the examples.

I am sure this fling would be very handy for you. If you have any doubts or need any help, please leave the comment.

Below are some of the very handy DRS flings

Cluster Rules Manager

DRS lens

DRS doctor

DRS dump insight

 

vSphere 6.7 : What is PMem and How vSphere DRS works with PMem configured VMs?

vSphere 6.7 supports one of the exciting technologies on computer memory i.e. Non-Volatile Memory (NVM) or persistent memory (PMem).  In this post my focus is on briefing what is PMem technology and explore how DRS works with PMem configured VMs.

Image is taken from VMware site
What is Persistent Memory (PMem)?

In brief, PMem is next generation memory technology whose data transfer speed is as good as DRAM (50-300 ns, 100 times faster than SSDs) and unlike DRAM, it can even retain the data after reboots .  If you ask me, it is going to greatly improve the performance of applications running performance sensitive workloads  ex. analytics and acceleration databases etc. The virtual machines, which require low latency, high bandwidth and at the same time persistence are right workloads to be benefited from this technology.

Below diagram highlights the 2 modes supported by vSphere.

PMem modes supported on vSphere

i) Direct-access mode (vPMem): It requires hw-version 14 and PMem-aware VMs ex. Windows server 2016 or RHEL 7.4.x

ii) Virtual Disk mode (vPMemdisk): This works with legacy/traditional VMs, no requirement of having hw version 14 and VM need not to be PMem aware.

If you want to understand more about PMem technology , I would recommend you to watch this youtube presentation by VMware’s CTO Richard B.

vSphere DRS interop with PMem VMs

First of all, I deployed one quick lab setup: 1 vCenter, 3 ESXi hosts and simulated PMem on 2 out of 3 hosts. All these 3 hosts are inside the DRS enabled cluster. Since I do not have any PMem aware VMs, I explored PMem in vPMemdisk mode i.e. Virtual Disk Mode pointed above

Note: I did simulate PMem on  2 hosts  using a cool hack provided by engineering i.e. by running this esxcli command “esxcli system settings kernel set -s fakePmemPct -v 33”. Note that simulation must be used for lab or educational purposes only. William has written a nice article on the same as well.

Below is how the PMem storage policy and PMem local datastores look like

 

Now that we have required PMem simulated hosts, lets start exploring how DRS works with PMem. Since I did not have any PMem aware guest, as I said earlier, I explored PMem with DRS in virtual disk mode described above i.e. vPMemDisk, which works fine for legacy guest OSes. Below are my observations on the same.

i) VM creation work-flow (vSphere H5 client)

I created a VM with PMem of size 512 MB on DRS cluster. Below screenshot shows how PMem can be selected as storage.

Below screenshot shows how PMem storage policy comes into picture. This policy gets selected automatically once you select “PMem” as shown above.

Now lets take a look at the VM summary.

ii) DRS initial placement
  • The VM I initially created got registered on one of the hosts with PMem storage and I observed that DRS places (once powered on) the VM on same host, where it was residing.
  • Then I created 2 PMem (same vmdk size) configured VMs and powered on. I see that DRS placed VMs on 2 separate hosts as expected.
iii) Host maintenance mode
  • I had 2 hosts with PMem and a host without PMem. I put one of the hosts with PMem into maintenance mode, where one of the PMem configured VMs was residing. I observed that DRS migrated the VM to another PMem host as expected.
  • Then I went ahead and put even 2nd host into maintenance mode. Since there is no any host with PMem by now, I observed 2nd host can not be put into maintenance mode.
iv) DRS Affinity rules
  • I played around both VM-VM and VM-host rules and I observed that DRS tries to satisfy rules but never migrates PMem configured VM to non PMem host even though rules gets violated.
v) DRS load balancing

I could not explore more on DRS load balancing with PMem VMs but what I have observed is that DRS prefers recommending non-PMem migrations more overs PMem VM migrations. If it is absolutely needed, only then DRS recommends PMem VM migration to other compatible hosts.

Notes on PMem
  • With vSphere 6.7, only one PMem local datastore is allowed per host.
  • Normal datastore operations are not available on PMem datastores and vSphere H5 client also does not list PMem datastores under regular datastore views. Only operation available is monitoring stats.
  • Monitor PMem datastore stats using ESXCLI 
  • Host-local PMem default storage policy can not be edited. Also, user can not change the PMem VM storage policy. In order to change, user either needs to migrate or clone that VM
  • PMem will be reserved no matter VM is powered on or not
  • Though PMem can be configured on standalone hosts. However, it is highly recommended to use DRS.
  • DPM (distributed power management) supports PMem as well.

Official documentation can be found  here & here

I hope you enjoyed this post. Please stay tuned for my another blog post on PMem, where I will discuss about creating PMem VMs using API.

 

 

 

 

Tutorial: Getting started with pyVmomi on Windows : Supports vSphere 6.7

Couple of months back, I wrote an article on “How did I get started with pyVmomi on ubuntu distro?”, which was very well received.  Recently I got several requests from readers  on how to set pyVmomi on Windows platform and I thought to write this tutorial, which will take you through steps required to set pyVmomi on my Windows 10.

Step 1: Install latest python for windows.

Latest 3.x python version available is 3.6.5 on python official portal . Download the python installer as pointed out in below screenshot.

Once downloaded, while installing python, make sure to select checkbox “Add python 3.6 to PATH” as shown below.

As soon as installation is completed, I checked environment variable and I see that python 3.6 path is set properly as shown below.

I see that python is installed as expected at directory i.e. C:\Users\shitolev\AppData\Local\Programs\Python\Python36, which is default directory. I also checked python version using “python -V” command and I see correct python version is installed.

Step 2: Install pyvmomi and pyvim python modules

As I already discussed in my pyVmomi on ubuntu article, we need “pip” to install these python modules. Note that “pip” itself gets installed as part of python installation. Let us go ahead for “pyvmomi” and pyvim” modules installation by running this command
“python.exe -m pip install pyvmomi pyvim” . You can run this command from any directory of your choice.

Below is how it looks like

you can see that latest “pyvmomi” i.e. the one supports vSphere 6.7  and “pyvim” python modules got installed successfully. I also see that these modules are installed at python  directory i.e. C:\Users\shitolev\AppData\Local\Programs\Python\Python36\Lib\site-packages

Now that we are done with setting up pyVmomi on Windows, its time to run a vSphere API sample to confirm everything is working fine.

Step 3: Running vSphere API sample.

For the sake of simplicity I chose the same sample I used in my ubuntu article

# VMware vSphere Python SDK, pyvmomi
# Simple script to get vCenter server product details

from pyVim.connect import SmartConnect
import ssl

s=ssl.SSLContext(ssl.PROTOCOL_SSLv23) # For VC 6.5/6.0 s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE

si= SmartConnect(host="10.192.1.2", user="Administrator@vsphere.local", pwd="VMware#123", sslContext=s)
aboutInfo=si.content.about

print ("Product Name:",aboutInfo.fullName)
print("Product Build:",aboutInfo.build)
print("Product Unique Id:",aboutInfo.instanceUuid)
print("Product Version:",aboutInfo.version)
print("Product Base OS:",aboutInfo.osType)
print("Product vendor:",aboutInfo.vendor)

We can keep this sample file at any location of our choice and execute it as follows.

Output shows that this small pyvmomi script got executed on vSphere 6.7 successfully. How cool is that?

Note: I would like you to take a look at line #7, where I had to use “ssl.PROTOCOL_SSLv23” protocol for VC 6.7 connection. For VC 6.0 and 6.5, I used “ssl.PROTOCOL_TLSv1”.

Useful resources

1. Please take a look at tutorial on how to get hold of vSphere objects and play around. Take a look at some cool samples on VC HA

2. If you want to contribute to pyVmomi, refer pyvmomi and its samples github repo

3. Not interested in Python vSphere SDK? no problem, take a look at vSphere Java SDK and PowerCLI posts.

4. Getting started with vCenter server REST APIs using python