Category Archives: Compute

posts on cpu/memory features/ server virtualization

Getting started with vCenter server REST APIs using python

You already might have noticed, as part of vSphere 6.5, VMware introduced vCenter Server REST APIs. I really enjoyed playing around them using vCenter apiexplorer as well as Postman REST client. Recently, I wanted to code around these APIs using one of the programming languages and I am happy that I was able to do it using Python. I thought it is worth to share with you. In this blog post, I will take you through all the steps required to get started with vCenter REST API using python. Here we go.

Step 1. First important thing is to get familiar with vCenter server REST API documentation. Similar documentation is available from vCenter apiexplorer as well. I would recommend you to play with apiexplorer, which will not only make you familiar with documentation but also will enable you to quickly invoke these APIs against your vCenter server. I am sure you will be pretty impressed on REST API documentation when compared to older vSphere SOAP API documentation.

Step 2. Once you explore the first step , we can get started with environment creation for REST APIs using python. If you ask me, it is fairly easy and quick, we just need to have python installed machine, where we need to install “requests” python module as follows

$ pip install requests

Step 3. Now let us take a look at below python module developed to simplify REST API usage.

# Author: Vikas Shitole
# Website: www.vThinkBeyondVM.com
# Product: vCenter server
# Description: Python module for vCenter server REST APIs
# Reference:https://code.vmware.com/apis/191/vsphere-automation
# How to setup vCenter REST API environment?: Just have VM with python and install "requests" python library using pip

import requests
import json
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

s=requests.Session()
s.verify=False

# Function to get the vCenter server session
def get_vc_session(vcip,username,password):
         s.post('https://'+vcip+'/rest/com/vmware/cis/session',auth=(username,password))
         return s

# Function to get all the VMs from vCenter inventory
def get_vms(vcip):
        vms=s.get('https://'+vcip+'/rest/vcenter/vm')
        return vms

#Function to power on particular VM
def poweron_vm(vmmoid,vcip):
        s.post('https://'+vcip+'/rest/vcenter/vm/'+vmmoid+'/power/start')

# Function to power off particular VM
def poweroff_vm(vmmoid,vcip):
        s.post('https://'+vcip+'/rest/vcenter/vm/'+vmmoid+'/power/stop')

Above vcrest.py module is available on my github repo.

Let us understand above code.

Line 8: Imported powerful “requests” python library required to make API calls
Line 9: Imported “json” library required to parse json response we get from REST APIs
Line 10/11: Here we are disabling warnings related to SSL connection. In production, we should not disable it.
Line 13/14: Here we are creating Session object to have session persisted during the current request. If you see “s.verify” is set to False, it does mean that we are ignoring verifying SSL certificates. If you want to set it to true, please take a look at SSL Cert Verification section
Line 16 to 32: I have added 4 methods i.e. get_vc_session(), get_vms(), poweron_vm() & poweroff_vm(). We would be calling these methods from below sample script. If you see, in all the methods, I have used REST API documentation and called these APIs using “requests” library.

Step 4. Now that we understood above “vcrest.py” module, let us import above module into our script to demonstrate its usage.

# Description: Python sample to get VMs and its moid using vCenter server REST API.
# Reference:https://code.vmware.com/apis/191/vsphere-automation
#  Make sure you have "rest.py" file into your python directory.

import vcrest
import json
vcip="10.192.23.143" # vCenter server ip address/FQDN

#Get vCenter server session and can be used as needed. pass vcenter username & password
vcsession = vcrest.get_vc_session(vcip,"Administrator@vsphere.local","VMware1!")

#Get all the VMs from inventory using below method from "vcrest" module.
vms = vcrest.get_vms(vcip)

# Parsing the JSON response we got from above function call (it has all the Vms present in inventory
vm_response=json.loads(vms.text)
json_data=vm_response["value"]

print "VM names and its unique MOID"
print "============================"
for vm in json_data:
        print vm.get("name")+" :: "+vm.get("vm")
        #We are powering on all the VMs those are in powered off state
        if vm.get("power_state") == "POWERED_OFF":
                vcrest.poweron_vm(vm.get("vm"),vcip)

Above script i.e. vcrestsample.py is available on my github repo as well

Output :
vmware@localhost:~$ python vcrestsample.py
VM names and its unique MOID
============================
NTP-India-1 :: vm-42
NTP-PA-2 :: vm-43
WebApp-1 :: vm-44
vThinkBeyondVM :: vm-45
vmware@localhost:~$

Let us understand above script.

Line 5: Imported “vcrest” module we just discussed above.
Line 10: We are getting vCenter server session by calling function defined in “vcrest” module. We can use this session object as needed.
Line 13: We are getting all the VMs from inventory using “get_vms() function defined in “vcrest” module. Note that with this call, we will get JSON response as shown below, which we need to parse to fetch useful information.

{
  "value": [
    {
      "memory_size_MiB": 512,
      "vm": "vm-42",
      "name": "NTP-India-1",
      "power_state": "POWERED_OFF",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-43",
      "name": "NTP-PA-2",
      "power_state": "POWERED_OFF",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-44",
      "name": "WebApp-1",
      "power_state": "POWERED_ON",
      "cpu_count": 1
    },
    {
      "memory_size_MiB": 512,
      "vm": "vm-45",
      "name": "vThinkBeyondVM",
      "power_state": "POWERED_ON",
      "cpu_count": 1
    }
  ]
}

Line 16/17: As we got JSON response as pointed above, here we parsed it so that we can easily access as part of python dictionary.
Line 21 to 25: Iterating through dictionary and printing vm names & its moid (managed object id). Finally powering on VMs those are off.

That is all, is not it cool? Since we have REST APIs available for vCenter VM life cycle, VCSA, content library, tagging etc, there is lot to learn and play around. I will keep adding more methods to vcrest.py module. If you are interested in contributing to this module, let me know, it would be really great. In case, you would like to explore vCenter SOAP based APIs, please refer my last post

pyVmomi tutorial : How to get all the core vCenter server inventory objects and play around?

Recently I posted about “how did I get started with pyvmomi?”. I continued exploring this further and I thought to share my learning so far. After setting up the pyvmomi environment, first thing I wanted to learn was to get hold of all core vCenter inventory objects so that I can play around each of them. vCenter core inventory includes objects such as Datacenters, Clusters, Virtual Machines, Hosts, Datastores & Networks etc.
As an initial step, I was looking for a way to get below objects from VC inventory
-All Virtual Machines
-All Clusters
-All Hosts
-All Datacenters

There could be multiple ways to get above objects but below is the way I could achieve this.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

# Get all the Vms from vCenter server inventory and print its name
# Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=si.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
getAllVms=get_all_objs(content, [vim.VirtualMachine])

#Iterating each vm object and printing its name
for vm in getAllVms:
        print vm.name

Output:
vmware@localhost:~$ python getAllVms.py
Database-server
NTP-Bangalore
Web-server
NTP-Pune
vmware@localhost:~$

You could see above, script listed all the VMs available in inventory. Let us look at another example, here we will get all the clusters from vCenter Inventory.

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl

#Get all the Clusters from vCenter invetory and printing its name
#Below is Python 2.7.x code, which can be easily converted to python 3.x version

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$h1vKamal",sslContext=s)
content=c.content

# Method that populates objects of type vimtype
def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

#Calling above method
clusters=get_all_objs(content, [vim.ClusterComputeResource])

#Iterating each cluster object and printing its name
for cluster in clusters:
        print (cluster.name)

Output:
vmware@localhost:~$ python getClusters.py
DRSCluster-USA
DRSCluster-Europe
DRSCluster-India
vmware@localhost:~$

If you notice, only important change from getAllVms and getClusters script is line #22, is not it cool? So now we were able to get all the VMs as well as all the clusters. Similarly we can get other core inventory objects such as datacenter, hosts, datastores, networks etc. Below is how line #22 will look for other vCenter/ESXi objects.

# For ESXi host
hosts=get_all_objs(content, [vim.HostSystem])

#For datacenters
dcs = get_all_objs(content, [vim.Datacenter])

#For datastores 
datastores = get_all_objs(content, [vim.Datastore])

By now we know how to get hold of vSphere inventory objects, let’s do something meaningful with these objects. I chose to call a method on one of clusters that we retrieved above. What is that method?
vCenter 6.0 has introduced one of interesting APIs on vSphere DRS cluster rules. i.e. findRulesForVM(). This method is called on individual cluster and it returns set of DRS VM-VM affinity rules (enabled or disabled) those are associated with given/input VM. Since this method returns VM-VM affinity rules, I first created some VM-VM rules on one of my clusters as follows.

Cluster with DRS enabled
DRS VM-VM rules

Now that DRS rules are created, let us pass VM name as parameter to “findRulesForVM()” API and call this API on that cluster.

Updated code is available on VMware sample exchange portal here

from pyVim.connect import SmartConnect
from pyVmomi import vim
import ssl
#Get DRS VM-VM rules associated with a VM.

s=ssl.SSLContext(ssl.PROTOCOL_TLSv1)
s.verify_mode=ssl.CERT_NONE
si= SmartConnect(host="10.192.3.2", user="Administrator@vsphere.local", pwd="$vabc325pqr",sslContext=s)
content=c.content

def get_all_objs(content, vimtype):
        obj = {}
        container = content.viewManager.CreateContainerView(content.rootFolder, vimtype, True)
        for managed_object_ref in container.view:
                obj.update({managed_object_ref: managed_object_ref.name})
        return obj

# Scanning a input VM inside inventory using special  python construct i.e. List comprehension
# It will get all the Vms and check whether input VM is available inside inventory or not, finally it returns list with matching condition

vmToScan = [vm for vm in get_all_objs(content,[vim.VirtualMachine]) if "NTP-Bangalore" == vm.name]

# Scanning a input cluster inside invetory the way we did for VM above. here also we used list comprehension.

cluster = [cluster for cluster in get_all_objs(content,[vim.ClusterComputeResource]) if "DRSCluster-India" == cluster.name]

# Now we can call the method on input cluster by passing input VM as parameter, it returns array of rule objects associated with input VM.
ClusterRuleInfo=cluster[0].FindRulesForVm(vmToScan[0])

# Now iterate through rule objects and print the rule name

for rule in ClusterRuleInfo:
        print rule.name

Output:
vmware@localhost:~$ python findRules.py
Anti-Affine-NTP-Servers
Affine-Web-DB-Servers

Here is how vSphere web client “recent task” looks, you could see “Find Rules for vms” task.

Notes:
-For the sake of simplicity, I have hard-coded some values and simplified the code, please make changes as per your requirement.
-I have added required documentation inside above code itself.
-This code follows mostly python 2.7.x version, please make minor changes to work with python 3.x

If you need any help, please let me know and yes, please stay tuned for my upcoming blog posts on pyVmomi.

vCenter Server 6.0U3c is live now: Some of cool improvements on Storage DRS

Today vCenter Server 6.0U3c is released. As per me, though this release is a patch release but looks like lot of improvements done on various vCenter/ESXi components. I had an opportunity to work on some of Storage DRS issues and I thought to share those with you. In fact, there are multiple improvements done from Storage DRS perspective but I am sharing some of very interesting ones. Below is the content from release notes with respect to Storage DRS.

1. Storage DRS might place the thin provisioned disks into one datastore, instead of distributing them evenly

During the initial placement of thin provisioned disks, the Storage Distributed Resource Scheduler (SDRS) might miscalculate the entitled space requirement by excluding the reserved space. As a result, SDRS might use only the committed megabytes for the entitled space calculation, causing a wrong placement recommendation on one datastore, instead of distributing them evenly. This issue is resolved in this release.

2. Storage DRS generates only one suitable datastore for initial virtual machine placement

In case there are virtual machines with Raw Device Mapping (RDM) virtual disks, the Storage DRS might consider the actual mapping file size instead of the pointer file size, even though it does not consume any disk space. As a result, the Storage DRS might generate only one suitable datastore when creating new virtual machines. This issue is resolved in this release.

3.vCenter Server might fail in attempt to create a virtual machine on a Storage DRS cluster using script with null CPU or Mem Share values

vCenter Server might fail if you attempt to create a virtual machine on a Storage DRS cluster using a script with null CPU or Mem Share values in a RecommendDatastores() API call. This issue is resolved in this release.

Based on above info, you might have got the high level insights on these Storage DRS improvements. If you ask me, all are really good improvements from Storage DRS perspective. In my future blog posts, I will have deep dive into each of the above issues.

I think, if you are using Storage DRS, it is one of the great reasons to upgrade your environment (apart from important fixes on other vCenter/ESXi areas)

Important links

Download vCenter Server 6.0U3c bits here

vCenter server 6.0 U3c release notes

Yes, there is corresponding ESXi release, please take a look at this KB

It seems, there are some great improvements made to VMware vSAN as well, please take a look at this KB

Editing Storage DRS VM overrides : Java vSphere SDK and PowerCLI script

Recently I had an opportunity to work on one of customers requirement with respect to Storage DRS. Their requirement was to edit/remove Storage DRS “VM overrides” settings using vSphere API. As part of this exercise, I had written scripts both in PowerCLI as well as using vSphere Java SDK. I thought it is good to share with you as well. Before discussing about these scripts, let us first understand what exactly is Storage DRS VM overrides?

When user selects Storage DRS cluster (SDRS POD) as a storage for a VM, Storage DRS takes care of placing that VM on right datastore among all the datastores inside Storage DRS cluster. Once SDRS places that VM, all the default SDRS cluster level configuration gets applied on that VM such as ‘Storage DRS automation level’, ‘Keeps VMDK together’ etc. “VM overrides” settings comes to into picture if user wants to override some of Storage DRS cluster level settings. Ex. SDRS cluster level settings for “Automation level” is “Manual” and now for particular VM, user wants to have it as “Fully Automated” or user wants to disable SDRS on particular VM or user just wants to disable default affinity rule “Keeps VMDK together” for specific VM. This can be achieved using SDRS VM overrides settings. Let us look at some of vSphere web client screenshot on how to do it.

Above screenshot shows how to traverse to the “VM overrides” workflow using web client. You could see, currently there is NO VM for which SDRS settings are overridden. You can click on add button in order to override SDRS settings for specific VM as shown below.

Apart from this, there is another reason VM can get listed under “VM orderride” section i.e. When user is creating a VM and user wants to place that VM on a particular datastore among datastores in SDRS cluster instead of relaying on SDRS itself for initial placement, it does mean that user would like to take control of this VM , hence SDRS gets disabled on such VM and gets listed under “VM override” section. Take a look at below screenshot to understand the same.

You could see in above screenshot that if user wants to place the VM on a particular datastore from SDRS cluster instead of relaying on SDRS, we can select specific datastore only when we select checkbox for disabling SDRS. Once VM creation is completed, you could see that VM gets listed under “VM overrides” section as shown below.

Now that we understood, what is VM overrides? and how to configure it? Please take a look at scripts for editing/removing these Storage DRS VM overrides. Below are script locations.

1. Java SDK script on my github repo and on VMware Sample Exchange

2. PowerCLI equivalent on my github repo and on VMware Sample Exchange

If you have still not setup your YAVI JAVA Eclipse environment:Getting started tutorial

Important tutorials to start with: Part I & Part II

If you want to understand Storage DRS, take a look at this whitepaper

Let me know if you have any comments.

VMworld 2017 session voting : Great opportunity to influence VMworld 2017 agenda

VMworld 2017 session voting is started couple of weeks back and if you ask me, this is great opportunity for us to influence what we would like to see at this mega event. I thought to just have one small post asking for your votes for some sessions we have submitted.

If you do not know what is VMworld: VMworld is a global conference for virtualization and cloud computing, hosted by VMware. It is the number-one event for enabling the digital enterprise. VMworld 2017 brings you innovation to accelerate your journey to a software-defined business—from mobile devices to the data center and the cloud. Every year VMware hosts this mega event both in US as well as Europe

This year, there are record 1300+ sessions submitted by VMware fraternity (VMware customers, partners, vExperts, Employees) on multiple areas such as Cloud Computing, End User Computing (EUC), Software Defined Datacenter (SDDC) and mobility management etc.

Below are the sessions we submitted. If you like these sessions,please make sure you vote before 13th April.

1. DRS Cluster Rules Manager :New configuration management approach [2175] (Breakout session: 60 min)

2. Quick talk on VMware Log Tuner (VLT) [2203] (Quick talk: 30 min)

3. Overview on DRS Cluster Rules Manager :New configuration management approach [2752] (Quick talk : 30 min):

Session 1 & 3 are about the fling that we recently released. i.e. Cluster Rules Manager . We would like to take the configuration management approach demonstrated by this fling to the several specialized use-cases from vCenter server.

Here are some of FAQs can help you on casting your valuable vote

Q: How do I cast my vote?
A: First, head over to the voting site: https://my.vmworld.com/scripts/catalog/uscatalog.jsp

Next:
1. Click on the each above link or Search for each sessions listed above, you can either search using session name or ID
2. To vote for a session, click on the star value (1-5 stars) on the Session ID to indicate your preference. off-course, we want 5 stars.
3. You’ll be prompted to log in (if not already logged in) to your VMworld account to have your vote recorded. (If you don’t have a VMworld account, you can set one up for free: https://www.vmworld.com/myvmworld-update!input.jspa)
4. Colored stars confirms that you have successfully voted.

Note: As per me, it is great to open VMworld account using our official email id.

Q:How many sessions can I vote on?
A:You can vote on as many sessions as you’d like. However, you may not vote multiple times for the same session.

Q: How does my vote impact the VMworld 2017 agenda?
A: Your vote will be considered by the VMware Content Team as they make decisions about 2017 session content. In addition, the VMworld 2017 program reserves space on the agenda for “customer choice” sessions that receive particularly high ratings. Tell a friend to vote, and you might see your favorite session on the agenda this August or September.

Q: Who can participate in session voting?
A: Session voting is open to everyone. You will need to log in to your vmworld.com account to participate. If you do not have a vmworld.com account, you can set one up for free. : https://www.vmworld.com/myvmworld-update!input.jspa

Complete VMworld 2017 Session voting FAQs