The problem is located inside the VM:
YYYY-MM-DD hh:mm:ss.xxx+02:00 <ERROR> [NNNN] VTools::waitForTask@vtools.cpp(496): An error occurred while saving the snapshot: Failed to quiesce the virtual machine.
The VM host receives a request from the Novastor DataCenter/Network via the VStorage API to deliver a consistent state of the VM.
To do this, the VM host first creates an inner snapshot of the VM at the VM OS level.
This is done using the VMWare tools.
When creating the inner snapshot, also called Quiescing of the VM, an error occurs and the backup is terminated with this error.
A common reason for this problem is the obsolescence or non-existence of VMware Tools on the VM. Please check this first. Is the service running in the VM?
Another reason that regular snapshots work but not with quiescing enabled (The NovaBACKUP backup client's VMware plugin backup method as well as the Virtual Dashboard backup method function WILL always attempt to queisce the VM by default, and in that case if queiscing does not work on that VM it will fail the backup; however NovaStor DataCenter has a GUI option to disable queiscing and CBT - Change Block Tracking) is due to a VSS provider registry key that exists but does not point to anything, hence the VSS app event viewer may state a file not found error and/or a provider not found error in the Application Event Viewer in the VM. To check that follow this 3rd party article: https://jackiechen.blog/2014/02/24/fail-to-quiesce-a-virtual-machine-while-taking-snapshot/. You will be required to be inside the Event Viewer in the Windows Logs > Application listing, and as soon as you attempt to snapshot that VM via your vCenter/ESXi to snapshot it with quiescing enabled, and it shows the error there, outside of even NovaBACKUP / DataCenter backup software attempting that snapshot, then refresh the events to see if you can track down a VSS error and then read the details of that error even to proceed to troubleshoot it.
Another reason may be the VSS in the VM.
Please check the VM's event log for system or application errors at the time listed in the log.
For VMs that are Oracle servers, for example, the event with the ID 44 can be displayed in the event display of the VM. In this case, check whether the Oracle server has activated the ARCHIVELOGS option. This is absolutely necessary so that the Oracle VSS Writer can shut down the application.
Another reason may be a VM connected (configured in the VM)
DVD or floppy drive. Please remove these drives from the VM configuration.
It is also possible that a legacy floppy drive is configured in the bios of the VM.
In this case proceed as recommended by VmWare to remove the floppy drive:
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1034616
Or the guest OS does not have enough disk space for the snapshot.
To do this in the guest with the Explorer->Mark Drive C->Right mouse button "Configure Shadow Copies"->Properties
look for the configured location.
Even if the vssadmin output shows that there is no space configured for shadow copies, the truth lies in the "Configure Shadow Copies". If something is configured there, increase this value by 50% and save it.
After that, the output of
vssadmin list shadowstorage
and "Configure Shadow Copies"
values match.
Finally, an unusually high number of VSS snapshots in the guest OS may be the reason that no more snapshots can be taken.
You can use
vssadmin list shadows
Display shadow copies on the guest OS and press
<font color="#ffff00">-==- sync:ßÇÈâÈâ
Delete shadow copies on the guest OS to free the space again.
Other reasons for Quiescing Error inside VM:
Windows service 'VMware Snapshot Provider' is not installed.
https://kb.vmware.com/s/article/2149640
Solution:
- Download VMware Tools Version 10.1.10 https://my.vmware.com/web/vmware/details?downloadGroup=VMTOOLS10110&productId=614
- Uninstall VMware Tools
- Install VMware Tools Version 10.1.10
Windows service 'VMware Snapshot Provider' runs permanently after reboot and prevents snapshots.
https://communities.vmware.com/thread/533736
Workaround:
Stop the 'VMware Snapshot Provider' service and set the startup type to manual if not already done.
3. Windows feature 'Windows Server Backup' is not installed.
https://kb.vmware.com/s/article/2068653
Solution:
- Windows Start -> Server Manager
- Manage -> Add Roles and Features
- Select Windows Server Backupmore
- Select Windows PowerShell
- Next -> Install -> Close
Further information about the cause of Quiescing Errors can be found in the vmware.log of the VM
/vmfs/volumes/Datastore/Affected_Virtual_Machine/vmware.log
Search for the word "quiesce" near the time of the backup error.
Attention, the time in vmware.log is GMT in contrast to the backup protocol.
You can find the file here:
VSphere GUI
-> Select Datastore
-> Select files tab
-> Select VM
-> download vmware.log
if the download is not possible: copy and download vmware.log
For more info on troubleshooting this read the following support KB article: https://support.novabackup.com/hc/en-us/articles/360005840073-VMware-Backup-Failed
For more info on troubleshooting VMware quiescing not working read the following VMware article: https://kb.vmware.com/s/article/1007696