The Ugly Truth About Thin Provisioning and The NTFS Silent Treatment

Thin provisioned storage can be used in 2 places:

  1. at the storage controller/LUN level and/or
  2. at a virtual guest drive level.

In virtual environments, there can even be a mix of both thin- and thick-provisioned storage. A virtual guest drive could be thick provisioned but the underlying storage could be thin-provisioned (known as thick on thin). Or, a virtual guest drive could be thin-provisioned while the underlying storage could be thick provisioned (known as thin on thick). A virtual guest drive could be thin-provisioned and the underlying storage could also be thin-provisioned (known as thin on thin).

The NTFS-Hypervisor-Storage Controller Disconnect

In a Windows environment, there can be a disconnect between the NTFS file system and the hypervisor and/or the storage controller. Despite the fact that a file has been deleted and NTFS now sees that space as free and no longer in use, the hypervisor and/or storage controller does not see that space as no longer in use and available to be reclaimed. This results in wasted storage, excess capacity and unnecessary expenditures.

Let’s go through an example.

A server has a drive that is seen by Windows as 1TB in size. While the drive is 1TB in size, the actual amount of space taken is actually far less. The drive currently has 100GB used and 900GB free. On Monday, 500GB of data is added. Windows now sees the drive as having 600GB used and 400GB free.

The storage controller and/or hypervisor expands the space used by the thin-provisioned drive in order to accommodate the 500GB of data added. Several months later, 500GB of data is deleted. Windows sees the drive as having 100GB used and 900GB free. However, the storage controller/hypervisor doesn’t know that Windows is no longer using this space and that this space can be reclaimed.

2 Methods to Reclaim Space

There are essentially 2 technologies used to reclaim wasted storage space: SCSI UNMAP and Zero Detect.

  1. SCSI UNMAP is a relatively new addition to the SCSI command set.
  2. Zero Detect detects when zeros are written to blocks/clusters.

Both technologies signal a storage controller or hypervisor that storage is no longer in use and can be reclaimed. Some environments may even support a mixture of both technologies. Depending on the storage controller and/or hypervisor in use, blocks/clusters are then reclaimed – either shrinking a thin-provisioned guest drive or returning space to the storage pool.

Depending on the hypervisor and/or storage controller in use, the process to actually shrink the virtual disk drive and/or reclaim the free space will either happen automatically or occur as a separate process that runs on the hypervisor or storage controller.

PerfectStorage provides an easier and more efficient way to reclaim this space. Rather than trying to use cumbersome, command-line tools – such as SDelete – on a Windows system, PerfectStorage automates the reclaim process. With PerfectStorage, you can schedule space reclamation to run at a specified date or at regular intervals. And this can all be done without requiring that system activity be minimized.

PerfectStorage: A simpler, more efficient way to reclaim space.

Category: PerfectStorageStorageVirtualization