pyVmomi tutorial : How to get all the core vCenter server inventory objects and play around?

Recently I posted about “how did I get started with pyvmomi?”. I continued exploring this further and I thought to share my learning so far. After setting up the pyvmomi environment, first thing I wanted to learn was to get hold of all core vCenter inventory objects so that I can play around each of them. vCenter core inventory includes objects such as Datacenters, Clusters, Virtual Machines, Hosts, Datastores & Networks etc.
As an initial step, I was looking for a way to get below objects from VC inventory
-All Virtual Machines
-All Clusters
-All Hosts
-All Datacenters

There could be multiple ways to get above objects but below is the way I could achieve this.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Get all the Vms from vCenter server inventory and print its name
# Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=si.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
getAllVms=get_all_objs(content, [vim.VirtualMachine])

#Iterating each vm object and printing its name
for vm in getAllVms:
        print vm.name

Output:
vmware@localhost:~$ python getAllVms.py
Database-server
NTP-Bangalore
Web-server
NTP-Pune
vmware@localhost:~$

You could see above, script listed all the VMs available in inventory. Let us look at another example, here we will get all the clusters from vCenter Inventory.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

#Get all the Clusters from vCenter invetory and printing its name
#Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=c.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
clusters=get_all_objs(content, [vim.ClusterComputeResource])

#Iterating each cluster object and printing its name
for cluster in clusters:
        print (cluster.name)

Output:
vmware@localhost:~$ python getClusters.py
DRSCluster-USA
DRSCluster-Europe
DRSCluster-India
vmware@localhost:~$

If you notice, only important change from getAllVms and getClusters script is line #22, is not it cool? So now we were able to get all the VMs as well as all the clusters. Similarly we can get other core inventory objects such as datacenter, hosts, datastores, networks etc. Below is how line #22 will look for other vCenter/ESXi objects.

# For ESXi host
hosts=get_all_objs(content, [vim.HostSystem])

#For datacenters
dcs = get_all_objs(content, [vim.Datacenter])

#For datastores 
datastores = get_all_objs(content, [vim.Datastore])

By now we know how to get hold of vSphere inventory objects, let’s do something meaningful with these objects. I chose to call a method on one of clusters that we retrieved above. What is that method?
vCenter 6.0 has introduced one of interesting APIs on vSphere DRS cluster rules. i.e. findRulesForVM(). This method is called on individual cluster and it returns set of DRS VM-VM affinity rules (enabled or disabled) those are associated with given/input VM. Since this method returns VM-VM affinity rules, I first created some VM-VM rules on one of my clusters as follows.

Cluster with DRS enabled
DRS VM-VM rules

Now that DRS rules are created, let us pass VM name as parameter to “findRulesForVM()” API and call this API on that cluster.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Get DRS VM-VM rules associated with a VM.

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$vabc325pqr",sslContext=s)
content=c.content

def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

# Scanning a input VM inside inventory using special  python construct i.e. List comprehension
# It will get all the Vms and check whether input VM is available inside inventory or not, finally it returns list with matching condition

vmToScan = [vm for vm in get_all_objs(content,[vim.VirtualMachine]) if "NTP-Bangalore" == vm.name]

# Scanning a input cluster inside invetory the way we did for VM above. here also we used list comprehension.

cluster = [cluster for cluster in get_all_objs(content,[vim.ClusterComputeResource]) if "DRSCluster-India" == cluster.name]

# Now we can call the method on input cluster by passing input VM as parameter, it returns array of rule objects associated with input VM.
ClusterRuleInfo=cluster[0].FindRulesForVm(vmToScan[0])

# Now iterate through rule objects and print the rule name

for rule in ClusterRuleInfo:
        print rule.name

Output:
vmware@localhost:~$ python findRules.py
Anti-Affine-NTP-Servers
Affine-Web-DB-Servers

Here is how vSphere web client “recent task” looks, you could see “Find Rules for vms” task.

Notes:
-For the sake of simplicity, I have hard-coded some values and simplified the code, please make changes as per your requirement.
-I have added required documentation inside above code itself.
-This code follows mostly python 2.7.x version, please make minor changes to work with python 3.x

If you need any help, please let me know and yes, please stay tuned for my upcoming blog posts on pyVmomi.

vCenter Server 6.0U3c is live now: Some of cool improvements on Storage DRS

Today vCenter Server 6.0U3c is released. As per me, though this release is a patch release but looks like lot of improvements done on various vCenter/ESXi components. I had an opportunity to work on some of Storage DRS issues and I thought to share those with you. In fact, there are multiple improvements done from Storage DRS perspective but I am sharing some of very interesting ones. Below is the content from release notes with respect to Storage DRS.

1. Storage DRS might place the thin provisioned disks into one datastore, instead of distributing them evenly

During the initial placement of thin provisioned disks, the Storage Distributed Resource Scheduler (SDRS) might miscalculate the entitled space requirement by excluding the reserved space. As a result, SDRS might use only the committed megabytes for the entitled space calculation, causing a wrong placement recommendation on one datastore, instead of distributing them evenly. This issue is resolved in this release.

2. Storage DRS generates only one suitable datastore for initial virtual machine placement

In case there are virtual machines with Raw Device Mapping (RDM) virtual disks, the Storage DRS might consider the actual mapping file size instead of the pointer file size, even though it does not consume any disk space. As a result, the Storage DRS might generate only one suitable datastore when creating new virtual machines. This issue is resolved in this release.

3.vCenter Server might fail in attempt to create a virtual machine on a Storage DRS cluster using script with null CPU or Mem Share values

vCenter Server might fail if you attempt to create a virtual machine on a Storage DRS cluster using a script with null CPU or Mem Share values in a RecommendDatastores() API call. This issue is resolved in this release.

Based on above info, you might have got the high level insights on these Storage DRS improvements. If you ask me, all are really good improvements from Storage DRS perspective. In my future blog posts, I will have deep dive into each of the above issues.

I think, if you are using Storage DRS, it is one of the great reasons to upgrade your environment (apart from important fixes on other vCenter/ESXi areas)

Important links

Download vCenter Server 6.0U3c bits here

vCenter server 6.0 U3c release notes

Yes, there is corresponding ESXi release, please take a look at this KB

It seems, there are some great improvements made to VMware vSAN as well, please take a look at this KB

How did I get started with the vSphere Python SDK “pyvmomi” on ubuntu distro?

I usually enjoy working on vSphere Java SDK & PowerCLI but recently I thought to explore pyVmomi. What is “pyVmomi”? It is an open source Python SDK for the VMware vSphere API that allows you to manage ESXi and vCenter server. I was able to get started with pyvmomi on ubuntu distro successfully and I thought it is worth to share the steps I performed with you. This guide is specific to setting up pyvmomi enviornment on ubuntu distro. Here we go.

On high level we need to have below prerequisites
– Ubuntu distro installed VM
– Python
– Pip

I installed ubuntu 16.04 and I could see python comes by default with ubuntu, thats good news. Here is how I checked ubuntu distro version and Python version

You can see, python version available is 3.5.2. Since we have python already available, next step is to install pip. What is pip? As per Wikipedia, pip is a package management system used to install and manage software packages written in Python.

There is a python script which installs pip module, hence first we need to download that python script i.e. get-pip.py as follows

Now that we have downloaded get-pip.py script, lets run it to install latest pip as follows

you could see, there was existing pip installation i.e. pip 8.1.1, which is uninstalled now and script installed latest pip-9.0.1. I think, it is better to have latest pip. Above step also confirms that our python environment is running fine.

Now we are all set with all prerequisites i.e. ubuntu, python and pip. Lets now install pyvmomi and pyvim modules. Please take a look at below screenshot.

As per above screenshot, pyVmomi, pyvim and all related modules got installed successfully.

Now its time to write a sample vSphere API script : I leveraged one of samples available in community & simplified it for your easy understanding.

# VMware vSphere Python SDK, pyvmomi
# Simple script to get vCenter server product details

from pyVim.connect import SmartConnect
import ssl
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.1.2", user="Administrator@vsphere.local", pwd="$h1vKamal", sslContext=s)
aboutInfo=si.content.about

print ("Product Name:",aboutInfo.fullName)
print("Product Build:",aboutInfo.build)
print("Product Unique Id:",aboutInfo.instanceUuid)
print("Product Version:",aboutInfo.version)
print("Product Base OS:",aboutInfo.osType)
print("Product vendor:",aboutInfo.vendor)

Output:
vmware@localhost:~$ python3 About.py
Product Name: VMware vCenter Server 6.5.0 build-5973321
Product Build: 5973321
Product Unique Id: ef56e4fd-5705-40da-93e7-d1cc57700d41
Product Version: 6.5.0
Product Base OS: linux-x64
Product vendor: VMware, Inc.
vmware@localhost:~$

Lets take a look at this sample script.
Line 1: pyVim is part of pyvmomi and is responsible for connection creation and deletion. What else it is responsible can be learned here
Line 5,6,7: These lines are responsible for handling SSL certificate stuff, more on this in later posts.
Line 8: It will get us a connection with vCenter or ESXi server. It will in fact get us the top level vSphere API object i.e. ServiceInstance. Please take a look at vSphere API reference.
Line 9 onwords: We are fetching a “about” property of “ServiceContent” object

To clone all the pyvmomi API samples repo into your local environment, run below command

Notes:
-You may have to deal with Python 2.7.x and 3.x syntax differences.
-If you want to make pyVmomi work with python 2.x version, just replace “python3” with “python” for commands used to install pyvim and pyvmomi modules. Also “pip3” with “pip”. My environment works for both python versions.

Important links
1. To have more fun with vSphere APIs, it is must that you should be friendly with vSphere API reference
2. To learn more about pyvmomi, refer pyvmomi and its samples github repo
3. Not interested in Python vSphere SDK? no problem, take a look at vSphere Java SDK and PowerCLI posts.

I am enjoying exploring further on pyvmomi, let me know about your experience. Please stay tuned for more pyvmomi API sample posts.

Editing Storage DRS VM overrides : Java vSphere SDK and PowerCLI script

Recently I had an opportunity to work on one of customers requirement with respect to Storage DRS. Their requirement was to edit/remove Storage DRS “VM overrides” settings using vSphere API. As part of this exercise, I had written scripts both in PowerCLI as well as using vSphere Java SDK. I thought it is good to share with you as well. Before discussing about these scripts, let us first understand what exactly is Storage DRS VM overrides?

When user selects Storage DRS cluster (SDRS POD) as a storage for a VM, Storage DRS takes care of placing that VM on right datastore among all the datastores inside Storage DRS cluster. Once SDRS places that VM, all the default SDRS cluster level configuration gets applied on that VM such as ‘Storage DRS automation level’, ‘Keeps VMDK together’ etc. “VM overrides” settings comes to into picture if user wants to override some of Storage DRS cluster level settings. Ex. SDRS cluster level settings for “Automation level” is “Manual” and now for particular VM, user wants to have it as “Fully Automated” or user wants to disable SDRS on particular VM or user just wants to disable default affinity rule “Keeps VMDK together” for specific VM. This can be achieved using SDRS VM overrides settings. Let us look at some of vSphere web client screenshot on how to do it.

Above screenshot shows how to traverse to the “VM overrides” workflow using web client. You could see, currently there is NO VM for which SDRS settings are overridden. You can click on add button in order to override SDRS settings for specific VM as shown below.

Apart from this, there is another reason VM can get listed under “VM orderride” section i.e. When user is creating a VM and user wants to place that VM on a particular datastore among datastores in SDRS cluster instead of relaying on SDRS itself for initial placement, it does mean that user would like to take control of this VM , hence SDRS gets disabled on such VM and gets listed under “VM override” section. Take a look at below screenshot to understand the same.

You could see in above screenshot that if user wants to place the VM on a particular datastore from SDRS cluster instead of relaying on SDRS, we can select specific datastore only when we select checkbox for disabling SDRS. Once VM creation is completed, you could see that VM gets listed under “VM overrides” section as shown below.

Now that we understood, what is VM overrides? and how to configure it? Please take a look at scripts for editing/removing these Storage DRS VM overrides. Below are script locations.

1. Java SDK script on my github repo and on VMware Sample Exchange

2. PowerCLI equivalent on my github repo and on VMware Sample Exchange

If you have still not setup your YAVI JAVA Eclipse environment:Getting started tutorial

Important tutorials to start with: Part I & Part II

If you want to understand Storage DRS, take a look at this whitepaper

Let me know if you have any comments.

My first ever VMworld 2017 experience.

This year, it was the very first time I got an opportunity to be part of VMworld Europe, Barcelona & I thought it would be good to share my experience with you. I am sure you will enjoy reading it.

1] My experience as Tech-talk Speaker

I had couple of tech-talks as part of VMworld vBrownBag. This was the very first time I got an opportunity to deliver tech-talks outside India & that too at biggest conference to unknown audience (Customers, partners etc). Personally, it was really a thrilling & exciting experience for me. More exciting was, tech-talks were arranged in open VMVillage community area (I was assuming that they would be in closed hall/room) & tech-talks were being live streamed/recorded as well. Based on the interest and feedback received, I could say both talks were really well received.

Below are the youtube links to my tech talks. Please take a look and please provide your feedback if any

i) VMware Log Tuner (aka vLog Tuner or VLT):
(Project contributors: @chiragarora, @durgakarri & @vThinkBeyondVM (myself))

Youtube link:

ii) DRS Cluster Rules Manager:
(Project contributors: @durgakarri, Gururja Hegdal & Vikas Shitole)

Youtube link :

Take a look at all vBrownBag tech-talks

2] Here are some of the VMworld highlights:

As usual, general sessions by our leaders were awesome. It was truly amazing to be part of such huge community & buzz. Apart from general sessions, as per me, below are the highlights.
i) VMware cloud on AWS (VMC or VMW on AWS): As expected, VMC had a lot of coverage across quick talks and breakout sessions. For me, the most insightful breakout session on VMC was by Solution Architect from Amazon and Technical marketing director from VMware i.e. “Business value of VMware Cloud on AWS”. In addition to these sessions, I attended some more insightful breakout sessions on VMC as well as other VMware products.

Here are some of top sessions from VMworld Europe

Here are some of top sessions from VMworld US

William Lam has compiled very nice list of all breakout sessions available on youtube

ii). Hands on Labs: After attending couple of sessions on VMC, I was very much interested in how this works for customer in action. This is where VMC on AWS HOL helped. It was truly great experience playing with very slick HTML5 based VMC interface & configuring various VMC components.

Here are more learning resources on VMware cloud on AWS

For learning VMware products, please look at Hands on lab

iii) Solution Exchange: This is the place where all of our partner ecosystem display their solutions. I visited 6-7 booths out of many others. It was heartening to see how VMware partner ecosystem has developed solutions around VMware products.

iv) Meeting with people: After communicating with many VMware & vExperts community members over email, blogs & twitter for last 3-4 years, it was exciting to meet some of VMware as well as vExperts community personally.

Selfie with Katie Bradely, vExpert Community manager

v) Things I missed: There are couple of things I missed as my tech-talks were scheduled on back-to-back days. However, as VMworld schedule is such huge, multiple tracks at the same time, we cannot catch everything as well. Having said that, I think, I missed to be part of VMworld Hackthon organized by our very own champs “William Lam”(@lamw) & “Alan Renouf”(@alanrenouf). I heard it was really cool (However, I did personally meet both of them on the way & visited VMware {code } booth to know all the programs they run). Another thing I missed is getting signed copy of this great & most sought book “VMware vSphere 6.5 Host Resources Deep Dive” by “Frank Denneman (@FrankDenneman) & Niels Hagoort (@NHagoort)”. Of course, I missed some of live breakout sessions but I can listen to recorded ones.

Here is blog post by William on VMworld Hackathon

Know more about VMware {code }

vi) Exploring Barcelona: Since BCN vs IST difference is just 3+ hrs, jet lag was not an issue & even temperature was similar what we have now in BLR. I explored BCN on the way back from VMworld Venue to hotel (5.5 KM) by walk, it was very pleasant experience. Yes, food was the issue but fortunately, I got “Indian Bollywood restaurant” walk-able distance from my hotel. In addition, being in Barcelona, I could feel the craze about football (During Barcelona vs Juventus match), how people worship players like Messi (Similar to Cricket in India). I would have liked visiting some places there but could not manage during my short trip.


vi) Parties/goodies & VMware store:
There was exclusive vExpert party arranged by our digital marketing team. Our own vSphere team has awarded all vExpert a Swag Bag containing “Power Bank, Book, Pen, bottle & some cool stickers”. In addition, many partners were giving away lot of goodies such as T-shirts, stickers, hat, bags etc. As I visited some of booths, I did get some of them & who does not like goodies? In addition, I enjoyed looking into VMware related books & official merchandise at VMware store. It was heartening to see the book that I reviewed as technical reviewer i.e. vSphere Design essentials.

vii) Other interesting booths: There were some more exciting booths/lounge such as “meet the experts” & “vmware educational services”.

Overall, I had a great trip, met many people, and learned a few things. Clearly, VMworld is a great platform to interact with VMware people, customers & vExperts