NovaBACKUP Virtual Dashboard VMware Differential Backup Retention issues
NovaBACKUP
Last Updated: Jan 09, 2015 02:27PM PST
This guide only applies to Virtual Dashboard VMware Differential Backups which have Retention enabled.
There are a couple of reasons why the NovaBACKUP Virtual Dashboard VMware Differential backups no longer have working retention (the oldest Differential backups are not being deleted in line with the specified retention level set in the existing Differential backup job). This guide also covers two reasons why backups processed via the Virtual Dashboard as a VMware VM Differential backup are no longer working to create new Differential backups:
First failure reason:
A. It is possible that the ESXi or vSphere server’s datastore for where Snapshots are stored is out of the necessary disk space to even have vSphere process a Snapshot of your VM which is what NovaBACKUP Virtual Dashboard calls when it attempts to backup a VM via Virtual Dashboard. It won’t know if vSphere does not have enough space to create a snapshot and won’t easily display an error to say such.
B. To investigate the actual debug log for why the NEW VM Differential backup is failing when it executes please view this log file by navigating with Windows Explorer to the folder located on the machine where Virtual Dashboard is installed on for example: "C:\ProgramData\NovaStor\NovaStor NovaBACKUP\Temp\"
Now look in this error log, after sorting the error logs by "Date Modified" in ascending order via Windows Explorer to see the latest "vmbkp-error*.log" files:
Open the latest vmbkp-error*log file with notepad or other text editor program and this will show you the actual ESX/vSphere VMware based error codes which go into much more depth than the Virtual Dashboard error code may be reporting this Differential VM backup error as. This will give you much better description of why the new VMware Differential backup is failing, possibly because of disk space on the ESXi/vSphere DataStore where the actual VM exists on the VMware side as to why Virtual Dashboard is failing to process the backup.
C. For the VM that can no longer process a new Differential backup of your VMware VM try to create a Snapshot of that VM via ESXi/vSphere server by right-clicking on the VM in question and create a new “Snapshot”. See if it saves correctly, if it doesn’t then it is probably disk space related or something on the datastore is not working properly on the ESXi/vSphere side.
Second failure reason and a workaround for future processing of retention:
Because you may have been manually deleting the folders that at least one of your VM Differential backups store themselves in for that VM backup there is still a file called “pool.db” that exists for each VM which stores the information on how many backups exist for that VM and where they are located for each one that it has run in the past. That file is not modified by you just deleting the folders where the backups are stored by your manual folder deletion:
“pool.db” is a database text file with the entries of all of the backups that this example VM has processed in the past. If you are backing up 4 VMware VM’s then you will have 4 x pool.db files in their own folder structure on your destination NAS.
Example "pool.db" location if your backups are going to a network path such as a NAS:
\\Destination of NovaBACKUP Virtual Dashboard stored backup (your NAS folder share)\NovaBACKUP\50293a12-2400-f991-9f73-b46dead4e0d4\pool.db
Example "pool.db" location if your backups are going to locally connected storage such as USB hard drive:
G:\Destination of NovaBACKUP Virtual Dashboard stored backup\NovaBACKUP\50293a12-2400-f991-9f73-b46dead4e0d4\pool.db
Since that file is not being modified after you do the manual folder deletion then it still thinks those other backups exist and could be breaking future backups of that VM. We have a retent.bat file to clear out those entries for you to create on the machine where Virtual Dashboard is installed on. Create a text file in notepad named "retent.bat" and store it in a simple named folder on the C: drive of that machine. You will be REQUIRED to modify the "retent.bat" example text contents by editing that file with notepad to change the GUID paths to match your environment for each VM you need to run it on to clear up the pool.db information. The pool.db file exists per VM that you are backing up, there is one pool.db for each VM in your list of VM’s you are backing up in the Virtual Dashboard.
The "retent.bat" batch/script file can process all of your VM’s pool.db files to clear up the number of retention entries per VM you are trying to backup. Here is the content of the "retent.bat" file for me to review with you on what would need to be edited to suit your unique environment:
In this example "retent.bat" content it is cleaning up two separate VM’s with a retention setting of two to only save two differential backup sets and always leave the full back up of each VM in place. I will explain the sections of the .bat below for you to know how to modify it to suit your environment:
"C:\Program Files (x86)\NovaStor\NovaStor NovaBACKUP\vmware\vmware.exe" cleanup --backupfolder "C:\Testing 4\NovaBACKUP\5036f3e5-6235-7698-f127-b06dfeefa993" --retent g=2 > C:\Temp\4-retentfolder-info.log 2> C:\Temp\4-retentfolder-error.log
"C:\Program Files (x86)\NovaStor\NovaStor NovaBACKUP\vmware\vmware.exe" cleanup --backupfolder "C:\Testing 6\NovaBACKUP\5036d7bd-3fce-2624-398c-549165df5b41" --retent g=2 > C:\Temp\6-retentfolder-info.log 2> C:\Temp\6-retentfolder-error.log
The first portion of each command calls vmware.exe located in the default location where it should be. This should not need to be changed in your case unless you are running a 32-bit OS or you installed NovaBACKUP to a non-standard folder.
“C:\Testing4\NovaBACKUP\5036f3e5-6235-7698-f127-b06dfeefa993" would need to be changed to suit your uniquely named folder where your VM is storing its backups. This is going to be unique per VM so for each VM you are specifying Virtual Dashboard to backup you will have a uniquely named “GUID” with that long string of numbers and text. Replace each of these portions of the line with the folder structure that matches in your case the VM you are needing to run the retention clean-up process on.
The “--retent g=2” tells it to keep TWO of your LATEST differential backups in place and wipe out the history and files for the older differential backups including editing the pool.db database file to reflect how many backups you actually now have in place after this cleanup retent.bat command executes. This command .bat file will never delete or clear up/delete the FULL backup it will only touch the Differential versions of the backup.
The last thing you would always need to modify in the script/bat is the folder and file where the two logs for the retention clean-up command are getting saved as. This will be unique to your environment and to mention which VM you are creating a clean-up log for. In our example the path is C:\Temp\4-retentfolder-info.log and C:\Temp\4-retentfolder-error.log. Change that portion of the .bat script to reflect what folder you are storing the retent clean-up log files in. The folder that you specify here has to actually EXIST prior to issuing the .bat script, the .bat won’t create the folder for you! You would also in the filename portion change the wording to suit which VM you are processing for clean-up.
On the computer with NovaBACKUP Virtual Dashboard installed on it save the MODIFIED retent.bat file (the one you actually modified to suit your environment) on a local C: drive path, possibly in a short folder path such as C:\TEMP. DO NOT attempt to execute the "retent.bat" file from a network path as it won’t work that way, the .bat has to be stored locally usually in a short path such as C:\TEMP. To run the retent.bat for the first time to make sure that it works properly start up an Administrative Command prompt on the computer that has the Virtual Dashboard installed on it. Via the already started Admin Command Prompt change to the folder path where the "retent.bat" is saved in and then execute the "retent.bat" by running retent.bat from the Admin Command Prompt. It should process very quickly and will produce two log files for you to check if items were processed successfully and which folders were deleted and retented out properly. After executing the retent.bat and waiting about 30 second open the "xx-retentfolder-info.log" retent log file where it exists as you modified it to exist in and that file will show you what items were dealt with. The "xx-retentfolder-error.log" retent log file will additionally show you any errors that came up when issuing the various retent.bat commands. Keep in mind that if the command was fully successful then the "xx-retentfolder-error.log" will actually be 0 bytes in size with no contents for you to read if no errors were produced by the retent.bat command.
This retent.bat method will give you the ability to not have to A. manually delete the Differential backups yourself for each VMware VM backup and B. should give you the ability to process NEW Differential backups going forward after the Differential backup folders are deleted (even if you manually deleted them). You can automate the retent.bat by creating a scheduled Windows Task Schedule entry for the retent.bat to be executed daily if needed so that no manual clean-up operations would need to be done or so that you don't have to remember to execute the retent.bat manually often enough. There are many tutorials on how to do this but we cannot support helping set that up. Here is a Google search result which gives you those tutorials to do this yourself on your own time: https://www.google.com/#q=windows+task+scheduler+tutorial.