Category Archives: HOME

Getting started with vCenter server REST APIs using python

You already might have noticed, as part of vSphere 6.5, VMware introduced vCenter Server REST APIs. I really enjoyed playing around them using vCenter apiexplorer as well as Postman REST client. Recently, I wanted to code around these APIs using one of the programming languages and I am happy that I was able to do it using Python. I thought it is worth to share with you. In this blog post, I will take you through all the steps required to get started with vCenter REST API using python. Here we go.

Step 1. First important thing is to get familiar with vCenter server REST API documentation. Similar documentation is available from vCenter apiexplorer as well. I would recommend you to play with apiexplorer, which will not only make you familiar with documentation but also will enable you to quickly invoke these APIs against your vCenter server. I am sure you will be pretty impressed on REST API documentation when compared to older vSphere SOAP API documentation.

Step 2. Once you explore the first step , we can get started with environment creation for REST APIs using python. If you ask me, it is fairly easy and quick, we just need to have python installed machine, where we need to install “requests” python module as follows

$ pip install requests

Step 3. Now let us take a look at below python module developed to simplify REST API usage.

# Author: Vikas Shitole
# Website: www.vThinkBeyondVM.com
# Product: vCenter server
# Description: Python module for vCenter server REST APIs
# Reference:https://code.vmware.com/apis/191/vsphere-automation
# How to setup vCenter REST API environment?: Just have VM with python and install "requests" python library using pip

import requests
import json
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

s=requests.Session()
s.verify=False

# Function to get the vCenter server session
def get_vc_session(vcip,username,password):
         s.post('https://'+vcip+'/rest/com/vmware/cis/session',auth=(username,password))
         return s

# Function to get all the VMs from vCenter inventory
def get_vms(vcip):
        vms=s.get('https://'+vcip+'/rest/vcenter/vm')
        return vms

#Function to power on particular VM
def poweron_vm(vmmoid,vcip):
        s.post('https://'+vcip+'/rest/vcenter/vm/'+vmmoid+'/power/start')

# Function to power off particular VM
def poweroff_vm(vmmoid,vcip):
        s.post('https://'+vcip+'/rest/vcenter/vm/'+vmmoid+'/power/stop')

Above vcrest.py module is available on my github repo.

Let us understand above code.

Line 8: Imported powerful “requests” python library required to make API calls
Line 9: Imported “json” library required to parse json response we get from REST APIs
Line 10/11: Here we are disabling warnings related to SSL connection. In production, we should not disable it.
Line 13/14: Here we are creating Session object to have session persisted during the current request. If you see “s.verify” is set to False, it does mean that we are ignoring verifying SSL certificates. If you want to set it to true, please take a look at SSL Cert Verification section
Line 16 to 32: I have added 4 methods i.e. get_vc_session(), get_vms(), poweron_vm() & poweroff_vm(). We would be calling these methods from below sample script. If you see, in all the methods, I have used REST API documentation and called these APIs using “requests” library.

Step 4. Now that we understood above “vcrest.py” module, let us import above module into our script to demonstrate its usage.

# Description: Python sample to get VMs and its moid using vCenter server REST API.
# Reference:https://code.vmware.com/apis/191/vsphere-automation
#  Make sure you have "rest.py" file into your python directory.

import vcrest
import json
vcip="10.192.23.143" # vCenter server ip address/FQDN

#Get vCenter server session and can be used as needed. pass vcenter username & password
vcsession = vcrest.get_vc_session(vcip,"Administrator@vsphere.local","VMware1!")

#Get all the VMs from inventory using below method from "vcrest" module.
vms = vcrest.get_vms(vcip)

# Parsing the JSON response we got from above function call (it has all the Vms present in inventory
vm_response=json.loads(vms.text)
json_data=vm_response["value"]

print "VM names and its unique MOID"
print "============================"
for vm in json_data:
        print vm.get("name")+" :: "+vm.get("vm")
        #We are powering on all the VMs those are in powered off state
        if vm.get("power_state") == "POWERED_OFF":
                vcrest.poweron_vm(vm.get("vm"),vcip)

Above script i.e. vcrestsample.py is available on my github repo as well

Output :
vmware@localhost:~$ python vcrestsample.py
VM names and its unique MOID
============================
NTP-India-1 :: vm-42
NTP-PA-2 :: vm-43
WebApp-1 :: vm-44
vThinkBeyondVM :: vm-45
vmware@localhost:~$

Let us understand above script.

Line 5: Imported “vcrest” module we just discussed above.
Line 10: We are getting vCenter server session by calling function defined in “vcrest” module. We can use this session object as needed.
Line 13: We are getting all the VMs from inventory using “get_vms() function defined in “vcrest” module. Note that with this call, we will get JSON response as shown below, which we need to parse to fetch useful information.

{
  "value": [
    {
      "memory_size_MiB": 512,
      "vm": "vm-42",
      "name": "NTP-India-1",
      "power_state": "POWERED_OFF",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-43",
      "name": "NTP-PA-2",
      "power_state": "POWERED_OFF",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-44",
      "name": "WebApp-1",
      "power_state": "POWERED_ON",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-45",
      "name": "vThinkBeyondVM",
      "power_state": "POWERED_ON",
      "cpu_count": 1
    }
  ]
}

Line 16/17: As we got JSON response as pointed above, here we parsed it so that we can easily access as part of python dictionary.
Line 21 to 25: Iterating through dictionary and printing vm names & its moid (managed object id). Finally powering on VMs those are off.

That is all, is not it cool? Since we have REST APIs available for vCenter VM life cycle, VCSA, content library, tagging etc, there is lot to learn and play around. I will keep adding more methods to vcrest.py module. If you are interested in contributing to this module, let me know, it would be really great. In case, you would like to explore vCenter SOAP based APIs, please refer my last post

pyVmomi tutorial : How to get all the core vCenter server inventory objects and play around?

Recently I posted about “how did I get started with pyvmomi?”. I continued exploring this further and I thought to share my learning so far. After setting up the pyvmomi environment, first thing I wanted to learn was to get hold of all core vCenter inventory objects so that I can play around each of them. vCenter core inventory includes objects such as Datacenters, Clusters, Virtual Machines, Hosts, Datastores & Networks etc.
As an initial step, I was looking for a way to get below objects from VC inventory
-All Virtual Machines
-All Clusters
-All Hosts
-All Datacenters

There could be multiple ways to get above objects but below is the way I could achieve this.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Get all the Vms from vCenter server inventory and print its name
# Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=si.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
getAllVms=get_all_objs(content, [vim.VirtualMachine])

#Iterating each vm object and printing its name
for vm in getAllVms:
        print vm.name

Output:
vmware@localhost:~$ python getAllVms.py
Database-server
NTP-Bangalore
Web-server
NTP-Pune
vmware@localhost:~$

You could see above, script listed all the VMs available in inventory. Let us look at another example, here we will get all the clusters from vCenter Inventory.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

#Get all the Clusters from vCenter invetory and printing its name
#Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=c.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
clusters=get_all_objs(content, [vim.ClusterComputeResource])

#Iterating each cluster object and printing its name
for cluster in clusters:
        print (cluster.name)

Output:
vmware@localhost:~$ python getClusters.py
DRSCluster-USA
DRSCluster-Europe
DRSCluster-India
vmware@localhost:~$

If you notice, only important change from getAllVms and getClusters script is line #22, is not it cool? So now we were able to get all the VMs as well as all the clusters. Similarly we can get other core inventory objects such as datacenter, hosts, datastores, networks etc. Below is how line #22 will look for other vCenter/ESXi objects.

# For ESXi host
hosts=get_all_objs(content, [vim.HostSystem])

#For datacenters
dcs = get_all_objs(content, [vim.Datacenter])

#For datastores 
datastores = get_all_objs(content, [vim.Datastore])

By now we know how to get hold of vSphere inventory objects, let’s do something meaningful with these objects. I chose to call a method on one of clusters that we retrieved above. What is that method?
vCenter 6.0 has introduced one of interesting APIs on vSphere DRS cluster rules. i.e. findRulesForVM(). This method is called on individual cluster and it returns set of DRS VM-VM affinity rules (enabled or disabled) those are associated with given/input VM. Since this method returns VM-VM affinity rules, I first created some VM-VM rules on one of my clusters as follows.

Cluster with DRS enabled
DRS VM-VM rules

Now that DRS rules are created, let us pass VM name as parameter to “findRulesForVM()” API and call this API on that cluster.

Updated code is available on VMware sample exchange portal here

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Get DRS VM-VM rules associated with a VM.

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$vabc325pqr",sslContext=s)
content=c.content

def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

# Scanning a input VM inside inventory using special  python construct i.e. List comprehension
# It will get all the Vms and check whether input VM is available inside inventory or not, finally it returns list with matching condition

vmToScan = [vm for vm in get_all_objs(content,[vim.VirtualMachine]) if "NTP-Bangalore" == vm.name]

# Scanning a input cluster inside invetory the way we did for VM above. here also we used list comprehension.

cluster = [cluster for cluster in get_all_objs(content,[vim.ClusterComputeResource]) if "DRSCluster-India" == cluster.name]

# Now we can call the method on input cluster by passing input VM as parameter, it returns array of rule objects associated with input VM.
ClusterRuleInfo=cluster[0].FindRulesForVm(vmToScan[0])

# Now iterate through rule objects and print the rule name

for rule in ClusterRuleInfo:
        print rule.name

Output:
vmware@localhost:~$ python findRules.py
Anti-Affine-NTP-Servers
Affine-Web-DB-Servers

Here is how vSphere web client “recent task” looks, you could see “Find Rules for vms” task.

Notes:
-For the sake of simplicity, I have hard-coded some values and simplified the code, please make changes as per your requirement.
-I have added required documentation inside above code itself.
-This code follows mostly python 2.7.x version, please make minor changes to work with python 3.x

If you need any help, please let me know and yes, please stay tuned for my upcoming blog posts on pyVmomi.

vCenter Server 6.0U3c is live now: Some of cool improvements on Storage DRS

Today vCenter Server 6.0U3c is released. As per me, though this release is a patch release but looks like lot of improvements done on various vCenter/ESXi components. I had an opportunity to work on some of Storage DRS issues and I thought to share those with you. In fact, there are multiple improvements done from Storage DRS perspective but I am sharing some of very interesting ones. Below is the content from release notes with respect to Storage DRS.

1. Storage DRS might place the thin provisioned disks into one datastore, instead of distributing them evenly

During the initial placement of thin provisioned disks, the Storage Distributed Resource Scheduler (SDRS) might miscalculate the entitled space requirement by excluding the reserved space. As a result, SDRS might use only the committed megabytes for the entitled space calculation, causing a wrong placement recommendation on one datastore, instead of distributing them evenly. This issue is resolved in this release.

2. Storage DRS generates only one suitable datastore for initial virtual machine placement

In case there are virtual machines with Raw Device Mapping (RDM) virtual disks, the Storage DRS might consider the actual mapping file size instead of the pointer file size, even though it does not consume any disk space. As a result, the Storage DRS might generate only one suitable datastore when creating new virtual machines. This issue is resolved in this release.

3.vCenter Server might fail in attempt to create a virtual machine on a Storage DRS cluster using script with null CPU or Mem Share values

vCenter Server might fail if you attempt to create a virtual machine on a Storage DRS cluster using a script with null CPU or Mem Share values in a RecommendDatastores() API call. This issue is resolved in this release.

Based on above info, you might have got the high level insights on these Storage DRS improvements. If you ask me, all are really good improvements from Storage DRS perspective. In my future blog posts, I will have deep dive into each of the above issues.

I think, if you are using Storage DRS, it is one of the great reasons to upgrade your environment (apart from important fixes on other vCenter/ESXi areas)

Important links

Download vCenter Server 6.0U3c bits here

vCenter server 6.0 U3c release notes

Yes, there is corresponding ESXi release, please take a look at this KB

It seems, there are some great improvements made to VMware vSAN as well, please take a look at this KB

How did I get started with the vSphere Python SDK “pyvmomi” on ubuntu distro?

I usually enjoy working on vSphere Java SDK & PowerCLI but recently I thought to explore pyVmomi. What is “pyVmomi”? It is an open source Python SDK for the VMware vSphere API that allows you to manage ESXi and vCenter server. I was able to get started with pyvmomi on ubuntu distro successfully and I thought it is worth to share the steps I performed with you. This guide is specific to setting up pyvmomi enviornment on ubuntu distro. Here we go.

On high level we need to have below prerequisites
– Ubuntu distro installed VM
– Python
– Pip

I installed ubuntu 16.04 and I could see python comes by default with ubuntu, thats good news. Here is how I checked ubuntu distro version and Python version

You can see, python version available is 3.5.2. Since we have python already available, next step is to install pip. What is pip? As per Wikipedia, pip is a package management system used to install and manage software packages written in Python.

There is a python script which installs pip module, hence first we need to download that python script i.e. get-pip.py as follows

Now that we have downloaded get-pip.py script, lets run it to install latest pip as follows

you could see, there was existing pip installation i.e. pip 8.1.1, which is uninstalled now and script installed latest pip-9.0.1. I think, it is better to have latest pip. Above step also confirms that our python environment is running fine.

Now we are all set with all prerequisites i.e. ubuntu, python and pip. Lets now install pyvmomi and pyvim modules. Please take a look at below screenshot.

As per above screenshot, pyVmomi, pyvim and all related modules got installed successfully.

Now its time to write a sample vSphere API script : I leveraged one of samples available in community & simplified it for your easy understanding.

# VMware vSphere Python SDK, pyvmomi
# Simple script to get vCenter server product details

from pyVim.connect import SmartConnect
import ssl
s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.1.2", user="Administrator@vsphere.local", pwd="$h1vKamal", sslContext=s)
aboutInfo=si.content.about

print ("Product Name:",aboutInfo.fullName)
print("Product Build:",aboutInfo.build)
print("Product Unique Id:",aboutInfo.instanceUuid)
print("Product Version:",aboutInfo.version)
print("Product Base OS:",aboutInfo.osType)
print("Product vendor:",aboutInfo.vendor)

Output:
vmware@localhost:~$ python3 About.py
Product Name: VMware vCenter Server 6.5.0 build-5973321
Product Build: 5973321
Product Unique Id: ef56e4fd-5705-40da-93e7-d1cc57700d41
Product Version: 6.5.0
Product Base OS: linux-x64
Product vendor: VMware, Inc.
vmware@localhost:~$

Lets take a look at this sample script.
Line 1: pyVim is part of pyvmomi and is responsible for connection creation and deletion. What else it is responsible can be learned here
Line 5,6,7: These lines are responsible for handling SSL certificate stuff, more on this in later posts.
Line 8: It will get us a connection with vCenter or ESXi server. It will in fact get us the top level vSphere API object i.e. ServiceInstance. Please take a look at vSphere API reference.
Line 9 onwords: We are fetching a “about” property of “ServiceContent” object

To clone all the pyvmomi API samples repo into your local environment, run below command

Notes:
-You may have to deal with Python 2.7.x and 3.x syntax differences.
-If you want to make pyVmomi work with python 2.x version, just replace “python3” with “python” for commands used to install pyvim and pyvmomi modules. Also “pip3” with “pip”. My environment works for both python versions.

Important links
1. To have more fun with vSphere APIs, it is must that you should be friendly with vSphere API reference
2. To learn more about pyvmomi, refer pyvmomi and its samples github repo
3. Not interested in Python vSphere SDK? no problem, take a look at vSphere Java SDK and PowerCLI posts.

I am enjoying exploring further on pyvmomi, let me know about your experience. Please stay tuned for more pyvmomi API sample posts.

Editing Storage DRS VM overrides : Java vSphere SDK and PowerCLI script

Recently I had an opportunity to work on one of customers requirement with respect to Storage DRS. Their requirement was to edit/remove Storage DRS “VM overrides” settings using vSphere API. As part of this exercise, I had written scripts both in PowerCLI as well as using vSphere Java SDK. I thought it is good to share with you as well. Before discussing about these scripts, let us first understand what exactly is Storage DRS VM overrides?

When user selects Storage DRS cluster (SDRS POD) as a storage for a VM, Storage DRS takes care of placing that VM on right datastore among all the datastores inside Storage DRS cluster. Once SDRS places that VM, all the default SDRS cluster level configuration gets applied on that VM such as ‘Storage DRS automation level’, ‘Keeps VMDK together’ etc. “VM overrides” settings comes to into picture if user wants to override some of Storage DRS cluster level settings. Ex. SDRS cluster level settings for “Automation level” is “Manual” and now for particular VM, user wants to have it as “Fully Automated” or user wants to disable SDRS on particular VM or user just wants to disable default affinity rule “Keeps VMDK together” for specific VM. This can be achieved using SDRS VM overrides settings. Let us look at some of vSphere web client screenshot on how to do it.

Above screenshot shows how to traverse to the “VM overrides” workflow using web client. You could see, currently there is NO VM for which SDRS settings are overridden. You can click on add button in order to override SDRS settings for specific VM as shown below.

Apart from this, there is another reason VM can get listed under “VM orderride” section i.e. When user is creating a VM and user wants to place that VM on a particular datastore among datastores in SDRS cluster instead of relaying on SDRS itself for initial placement, it does mean that user would like to take control of this VM , hence SDRS gets disabled on such VM and gets listed under “VM override” section. Take a look at below screenshot to understand the same.

You could see in above screenshot that if user wants to place the VM on a particular datastore from SDRS cluster instead of relaying on SDRS, we can select specific datastore only when we select checkbox for disabling SDRS. Once VM creation is completed, you could see that VM gets listed under “VM overrides” section as shown below.

Now that we understood, what is VM overrides? and how to configure it? Please take a look at scripts for editing/removing these Storage DRS VM overrides. Below are script locations.

1. Java SDK script on my github repo and on VMware Sample Exchange

2. PowerCLI equivalent on my github repo and on VMware Sample Exchange

If you have still not setup your YAVI JAVA Eclipse environment:Getting started tutorial

Important tutorials to start with: Part I & Part II

If you want to understand Storage DRS, take a look at this whitepaper

Let me know if you have any comments.