VMware releases VSAN and the power of policies

VMware Virtual SAN (VSAN)
Diagram by VMware

VMware is pushing back on storage vendors as it seeks to show the world the full power of software-defined storage.  A couple weeks ago, VMware announced the VMware Virtual SAN (VSAN), a new software-defined storage platform tightly coupled with vCenter server.   The new platform requires both hard disks and solid state disks to create a pool of redundant, shared storage within a cluster.  The hard disks are the core of the storage and the SSD are used for buffering and read caching but do not contribute to the total amount of storage.  And although the name has SAN in it, the traditional SAN that is bread and butter of its parent company, EMC, could not be more different than VSAN.

In many ways, this is similar to the VMware Virtual Storage Appliance, but VSAN is much more integrated into vCenter and the vSphere platform.  The requirement of SSD is also new and will greatly contribute to overall responsiveness for the storage.  But more importantly, the new software defined storage platform is an example of the possibilities yet to come from tightly integrating storage with vSphere.

The real power of what VMware has released is not the storage technology itself or the simplicity that it can be deployed (single-click simplicity like HA and DRS).  The real power is the addition of Storage Policies completely throughout vSphere and VSAN.  The policies are the real capability that can be leveraged across multiple types of storage.  As I said earlier, I think VMware is simply showing all storage vendors this is what is possible.  The VSAN is certainly not a SAN replacement for most enterprise customers, but there are certainly remote office or small office use cases where this will fit nicely without the need of purchasing a purpose built storage array — use cases, by the way, where virtual storage appliances (VSA) currently fit.

But back to the policies.  Introduced in vSphere 5.0, Storage Policies allow administrators to tag and set classing of storage for different pools that are available within a vSphere cluster.  The Storage Policies allow an administrator to abstract the back-end storage and provide a list of capabilities to end-users or operators who are creating new virtual machines – and choose a policy during initial deployment to meet their needs.  You see, what VMware has discovered is that the end users don’t care if the back-end storage is a 3PAR or a VMAX – a Synology or a Drobo.  What they care about is a class of service provided, response times and how those translate to their application and its performance.  They also care about the cost of charge-back, if an organization has implemented those types of billing scenarios.  Policies are all a way of defining those attributes back to the end user and abstracting the actual storage from those decisions.

I am sure many VMware customers are like myself.  When I started the virtualization project, most systems ran on local storage.  The SAN was reserved for our most mission critical systems and most of those were clustered.  Today, however, vSphere has moved all of those workloads onto the SAN.  It has increased my resiliency and ability to survive a disaster, but its also driven storage costs up exponentially.  Yes, VMware is owned by EMC – goal to sell disks accomplished.

But the VSAN must run a bit counter to the EMC culture, so back to my original point.  In many ways, I think the VSAN is simply a proof of concept of what is possible with software defined storage.  It is a way for VMware to show the rest of the world what we can do.  VMware is very good at fostering its ecosystem and partner vendors, so I think this will spur new innovation.

And so, for a third time, back to policies.  With the introduction of VSAN, VMware is extending this capability of Storage Policies to allow administrators to assign a policy to a VM on creation and then the policy lives with the VM allowing policy based management of the VM over time.  Not only can it be setup initially with the VM, it can follow and adapt as the back-end storage changes over time.  With the addition of storage pools and maintenance mode for storage, VMware has automated a lot of the manual labor around managing storage.  By creating the pools of storage and moving datastores in and out of them, administrators and move or remove storage without interrupting service of virtual machines.  And also by applying tags and policy around those storage pools, it allows for the addition and removal of datastores as required to onramp or offramp storage as purchasing or lease cycles dictate.  All of this while maintaining service levels and agreements for customers and workloads – whether those customers are internal or external to your organization.

Beyond backend maintenance of storage, say the attributes of a particular application or VM changes over time.  Say it now requires a higher tier of storage – well thats an easy policy change as well on the VM and with the magic of storage vMotion and fully-automated storage DRS, the vSphere platform will perform the necessary moves to increase performance and move the workload onto the appropriate storage de jour.

In many ways, this is all done with the idea of reducing the menial tasks that an administrator has to perform.  Its mean to kill the tracking spreadsheets and other mechanisms we have traditionally relied upon for classing storage or moving workloads appropriately.  It all goes to making the storage layer of vSphere smarter and more automated.

Now, the last piece of policy that really sets VSAN apart from the competition.  Within VSAN, the vSphere administrator can dictate the type of RAID, the number of stripes and copies that are present for a particular VM or datastore.  For the first time, the core vSphere platform is controlling storage down the stack and automating how it operates.  vSphere policy now defines where and how replication occurs, how and where stripes of data are implemented, and on a per VM basis, quality of service attributes to prevent noisy neighbor problems.  And combining those together gives context and visibility up the stack in vSphere, too.  The two way communication of the policy and the storage implementation means that the administrator can more granularly dictate how the storage should act to support the requirements of the virtual workload.

So, I have to say, after a week in San Francisco at VMworld 2013, the VSAN is the one piece I am most excited to test and evaluate.  If for no other reason, you should sign up for the VSAN beta to see where the future of storage is going.  I think the writing is on the wall of every vendor is talking software-defined, the VSAN represents the first software-defined storage product with such a deep level of integration into vSphere.  I fully expect to see other vendors like Nexenta and HP StoreVirtual quickly adopting this paradigm – maybe even making it to market with product offerings around the same time that VSAN becomes generally available.  API calls and other integration points should be very simple for software storage implementations.  I’m excited to see where this next step takes us.