Crucial fixes on Storage DRS in VMware vCenter Server 5.5 U2 : Part-I

HOME

There are a couple of crucial bug fixes on Storage DRS (Storage Distributed Resource Scheduler) in vCenter Server 5.5 U2. I thought it is worth to have article on the same. Here we go on first fix.

Issue is as follows:

1. When Virtual Machine level backup is in progress, by design, backup vendors leverage 2 important VDDK( Virtual Disk Development Kit)  APIs in order to inform vCenter  managed host  about backup is in progress.

2. PrepareForAccess() VDDK API is used to disable storage vMotion operation on the VM to be backed & EndAccess() VDDK API is used to enable storage vMotion operation once the backup is over. Refer: VDDK APIs to disable_Enable storage vMotion

3. When VM is NOT managed by SDRS or when SDRS is disabled on VM to be backed, there was not any issue and vCenter was working as expected on disabling and enabling storage vMotion when VM level backup is in progress.

4. BUT when VM is managed by SDRS cluster & even though backup solution was successfully invoking PrepareForAccess() VDDK API to disable storage vMotion, SDRS was not honoring the disabled storage vMotion and it was automatically starting storage vMotion may be based on space/IO load , affinity rule constraints or Maintenance mode operations. Due to this, backup in progress for that VM was getting failed and orphaned virtual machine disks (VMDKs) were getting created in virtual machine folder. Note that SDRS will automatically apply the storage vMotion recommendations when set at Fully Automated mode.

5. Even manual storage vMotion was allowed for the VM to be backed up from VI client “Migrate” Option and Web client “Move to” Option. This was also leading to same issue.

Resolution :

When VM is managed by SDRS cluster, below are the scenarios in which VM can be storage vMotion to other datastore in SDRS cluster.

1. Manual storage migration scenario.

2. Enter SDRS maintenance mode scenario.

3. Scenario around initial placement.

4. All run time placement scenarios.

5. All scenarios around SDRS affinity/anti-affinity rules.

With the fix, in any of above scenarios , SDRS & you are NOT allowed to storage vMotion VM from a datastore to other datastore (inside or outside SDRS cluster) when backup is in progress. Now apart from backup, even though storage vMotion is disabled for some other purpose, SDRS& you are not allowed to storage vMotion the VM.

My testing:

Note: Instead of using actual backup solution, I directly used above specified VDDK APIs to disable/enable storage vMotion on the input VM as same APIs are being used by backup solutions.

  • I tested this fix and I could see fix is working perfectly fine with NO any issue. Here is where (refer below screen shot) you can verify whether storage vMotion is disabled or not using MOB i.e. Managed Object Browser. (Refer:How to navigate MOB)

Disabled method from MOB

“vim.VirtualMachine.relocate” method is leveraged by storage vMotion.

  •   I tried putting datastore where VM is placed in maintenance mode (from SDRS cluster), I was not allowed to put DS in maintenance mode as expected. (Refer: below screenshot) SDRS maintenance mode
  • I added one more VMDK to the VM and created SDRS VMDK anti-affinity rule on VM which is being backed up. Ideally in order to satisfy  SDRS VMDK anti-affinity rule, one of VMDKs SDRS should storage vMotion to other datastore in SDRS cluster but SDRS throws fault as expected (refer: below screenshot)VMDK _Anti_affinity
  • I tried manual storage vMotion of the VM to be backed up but my attempt failed as expected. (Refer: below screenshot)Manual Migration

 

If you want to avoid such issue in your environment, you will have to upgrade to vCenter server 5.5 U2 . Download :vCenter 5.5 U2