Tag:

VMworld 2013

VMware releases VSAN and the power of policies

by Philip Sellers September 19, 2013

written by Philip Sellers

VMware Virtual SAN (VSAN)
Diagram by VMware

VMware is pushing back on storage vendors as it seeks to show the world the full power of software-defined storage. A couple weeks ago, VMware announced the VMware Virtual SAN (VSAN), a new software-defined storage platform tightly coupled with vCenter server. The new platform requires both hard disks and solid state disks to create a pool of redundant, shared storage within a cluster. The hard disks are the core of the storage and the SSD are used for buffering and read caching but do not contribute to the total amount of storage. And although the name has SAN in it, the traditional SAN that is bread and butter of its parent company, EMC, could not be more different than VSAN.

In many ways, this is similar to the VMware Virtual Storage Appliance, but VSAN is much more integrated into vCenter and the vSphere platform. The requirement of SSD is also new and will greatly contribute to overall responsiveness for the storage. But more importantly, the new software defined storage platform is an example of the possibilities yet to come from tightly integrating storage with vSphere.

The real power of what VMware has released is not the storage technology itself or the simplicity that it can be deployed (single-click simplicity like HA and DRS). The real power is the addition of Storage Policies completely throughout vSphere and VSAN. The policies are the real capability that can be leveraged across multiple types of storage. As I said earlier, I think VMware is simply showing all storage vendors this is what is possible. The VSAN is certainly not a SAN replacement for most enterprise customers, but there are certainly remote office or small office use cases where this will fit nicely without the need of purchasing a purpose built storage array — use cases, by the way, where virtual storage appliances (VSA) currently fit.

But back to the policies. Introduced in vSphere 5.0, Storage Policies allow administrators to tag and set classing of storage for different pools that are available within a vSphere cluster. The Storage Policies allow an administrator to abstract the back-end storage and provide a list of capabilities to end-users or operators who are creating new virtual machines – and choose a policy during initial deployment to meet their needs. You see, what VMware has discovered is that the end users don’t care if the back-end storage is a 3PAR or a VMAX – a Synology or a Drobo. What they care about is a class of service provided, response times and how those translate to their application and its performance. They also care about the cost of charge-back, if an organization has implemented those types of billing scenarios. Policies are all a way of defining those attributes back to the end user and abstracting the actual storage from those decisions.

I am sure many VMware customers are like myself. When I started the virtualization project, most systems ran on local storage. The SAN was reserved for our most mission critical systems and most of those were clustered. Today, however, vSphere has moved all of those workloads onto the SAN. It has increased my resiliency and ability to survive a disaster, but its also driven storage costs up exponentially. Yes, VMware is owned by EMC – goal to sell disks accomplished.

But the VSAN must run a bit counter to the EMC culture, so back to my original point. In many ways, I think the VSAN is simply a proof of concept of what is possible with software defined storage. It is a way for VMware to show the rest of the world what we can do. VMware is very good at fostering its ecosystem and partner vendors, so I think this will spur new innovation.

And so, for a third time, back to policies. With the introduction of VSAN, VMware is extending this capability of Storage Policies to allow administrators to assign a policy to a VM on creation and then the policy lives with the VM allowing policy based management of the VM over time. Not only can it be setup initially with the VM, it can follow and adapt as the back-end storage changes over time. With the addition of storage pools and maintenance mode for storage, VMware has automated a lot of the manual labor around managing storage. By creating the pools of storage and moving datastores in and out of them, administrators and move or remove storage without interrupting service of virtual machines. And also by applying tags and policy around those storage pools, it allows for the addition and removal of datastores as required to onramp or offramp storage as purchasing or lease cycles dictate. All of this while maintaining service levels and agreements for customers and workloads – whether those customers are internal or external to your organization.

Beyond backend maintenance of storage, say the attributes of a particular application or VM changes over time. Say it now requires a higher tier of storage – well thats an easy policy change as well on the VM and with the magic of storage vMotion and fully-automated storage DRS, the vSphere platform will perform the necessary moves to increase performance and move the workload onto the appropriate storage de jour.

In many ways, this is all done with the idea of reducing the menial tasks that an administrator has to perform. Its mean to kill the tracking spreadsheets and other mechanisms we have traditionally relied upon for classing storage or moving workloads appropriately. It all goes to making the storage layer of vSphere smarter and more automated.

Now, the last piece of policy that really sets VSAN apart from the competition. Within VSAN, the vSphere administrator can dictate the type of RAID, the number of stripes and copies that are present for a particular VM or datastore. For the first time, the core vSphere platform is controlling storage down the stack and automating how it operates. vSphere policy now defines where and how replication occurs, how and where stripes of data are implemented, and on a per VM basis, quality of service attributes to prevent noisy neighbor problems. And combining those together gives context and visibility up the stack in vSphere, too. The two way communication of the policy and the storage implementation means that the administrator can more granularly dictate how the storage should act to support the requirements of the virtual workload.

So, I have to say, after a week in San Francisco at VMworld 2013, the VSAN is the one piece I am most excited to test and evaluate. If for no other reason, you should sign up for the VSAN beta to see where the future of storage is going. I think the writing is on the wall of every vendor is talking software-defined, the VSAN represents the first software-defined storage product with such a deep level of integration into vSphere. I fully expect to see other vendors like Nexenta and HP StoreVirtual quickly adopting this paradigm – maybe even making it to market with product offerings around the same time that VSAN becomes generally available. API calls and other integration points should be very simple for software storage implementations. I’m excited to see where this next step takes us.

September 19, 2013 0 comments

Virtualization

HP Virtual Performance Viewer offers heat map for virtual performance

by Philip Sellers September 11, 2013

written by Philip Sellers

A couple weeks ago at VMworld, I stopped by the HP booth to see what they had new for this year and I was introduced to HP Virtualization Performance Viewer (vPV). Presented as a virtual triage tool, the idea behind vPV is to quickly let a virtualization administrator identify host spots of CPU or Memory that deviate from their baseline so they can pinpoint and fix problems quickly.

vPV comes in two editions – a free edition supporting up to 200 VM objects and a paid for edition with no limit.

The interface is clean and shows an overview of the entire environment, color coded – with shades of greens being good and oranges and red representing the problem areas. What is even better is you are able to drill down into a specific host or VM by double clicking one of the blocks. Drill down is simple and acts as it should, filtering the results to a specific cluster, resource pool, datastore or host to view what is happening inside of that object. This provides quick information about why a particular object is deviating from its norm.

A simple toggle radio button on the right side allows users to switch between vSphere and Hyper-V in the interface.

Screenshot of the HP Virtualization Performance Viewer dashboard — click for larger image

Hypervisor Support

vPV documentation says the system supports both VMware vSphere, Microsoft Hyper-V, KVM and Xen hypervisors. In addition, it supports OpenStack OS in a private cloud deployment and HP Cloud Application Platform as a Service (Paas).

In the free edition that I downloaded, I was able to see where I could configure against VMware vCenter or Microsoft System Center Virtual Machine Manager, but not against the other hypervisors or hosts listed. vCenter requires no agent. For Hyper-V, it appears SCVMM is required and to communicate with SCVMM an additional data collector must be installed. I did not see a requirement for SCVMM in the documentation, however.

HP is very big on the message of customer choice and support across heterogeneous virtualized environments. This is another product supporting the message, in my opinion.

Installation

HP documentation says that deployment should take about 15 minutes and I found that to be true. vPV is packaged as a virtual appliance, so the installation into VMware was quick and simple from an OVA template. After spinning up the OVA, configuration against vCenter was simple, too, except I had too many VM objects for the free edition and so vPV presented an error. I was able to begin testing against a subset of VMs by creating a special username and limiting permissions to specific hosts, so the free tool can be useful even in a larger environment against a subset of virtual guests.

Free versus Licensed

The free edition of vPV does not provide any security for the application. If you can get the URL for your browser, you are in and able to make configuration changes. The licensed edition includes AD integration and password protection for the application. The licensed edition also includes 3 months of data retention, while the free version only includes 1 day.

Integration

The licensed version of vPV has integration points against other HP management software packages – HP Performance Manager and HP Business Software Manager (BSM).

Support

For the free edition, community support is available from an HP support forum.

Video Overview – How to install vPV in under 15 minutes

More Information

September 11, 2013 0 comments

Datacenter Storage Virtualization

VMware unwraps vSphere 5.5 – What’s in it for your Datacenter?

by Philip Sellers September 9, 2013

written by Philip Sellers

Almost two weeks ago now, VMware unwrapped vSphere 5.5 to the public. The release, which ships later this year, includes a lot of enhancements and some new features for the datacenter. I won’t lie – 5.5 has a lot more in it for the datacenter than 5.1 enhancements. And I left VMworld 2013 much more excited about the direction of the virtual datacenter than I did at the end of VMworld 2012. So what are the things that sys admins will really care about in vSphere 5.5?

Expected enhancements – bigger maximums and more Web Client

As expected, the new release of vSphere increases maximums. The ESXi host maximums were doubled from vSphere 5.1 to vSphere 5.5 in all cases, except for the maximum LUN size, which got a surprising 32X bump in limit to 64TB. The number of logical CPU’s doubled to 320, the number of virtual CPU’s doubled to 4096 and the amount of RAM has doubled to 4TB per host. Those represent some pretty incredible numbers. vSphere 5.5 also introduces Virtual Hardware version 10. The new hardware version is introduced to support newer CPU architectures as well as a new virtual-SATA Advanced Host Controller Interface (AHCI) with up to 4 controllers and 30 devices per controller. So, the new AHCI supports 120 total devices connected to a single VM.

Also expected, vSphere 5.5 drives users more towards into the vSphere Web Client for management. All new capabilities are controlled from the vSphere Web Client. This follows suit with vSphere 5.1, whose new capabilities like shared-nothing vMotion, were available only in the Web Client. One thing I am hearing repeatedly on social media is that the vSphere 5.5 Web Client is noticeably more responsive, which is great. I am also hearing the new Virtual Hardware 10 VMs can only be managed through the Web Client, which could cause heartburn for stand-alone hosts, but the final release may change this functionality. The beta version of the Windows VI Client throws an error if you try to edit a Virtual Hardware 10 VM. The error says the version 10 VM must be managed through the Web Client. This just means that resiliency of the vCenter servers continue to be critical parts of virtual deployments as it is a required part of managing the infrastructure.

And speaking of vCenter Server, the vCenter Server Virtual Appliance (VCSA) continues to become a more capable alternative to a Windows server deployment of vCenter. With vSphere 5.5, the VCSA has a retooled embedded Postgres database capable of supporting 100 hosts and 3000 VMs, although those numbers may actually change for better or worse. During one Ask the Expert session during the week, a member of the VMware Labs team said they were using a single VCSA to support as many as 500 hosts, but mileage my vary. This is one of those numbers that may change during final release.

I’ll file SSO enhancements under expected as well. While most people outside of VMware did not know what VMware intended to do with SSO, one session at VMworld pretty much summed it up by saying that the new version of vSphere SSO releasing with vSphere 5.5 will be what should have been released the first time. To many adminstrators, myself included, the version of SSO introduced with vSphere 5.1 just didn’t seem up to VMware standards. It seemed to be a third party tool that was included but not fully streamlined into the vSphere implementation.

The vSphere 5.5 SSO version includes a number of streamlined improvements. AD integration is much improved and includes the ability to tie it to multiple forests, which is needed for many larger environments – or anywhere a merger has occurred. There is no longer a need for an external database. And there is a multi-master replication model built into this new version.

Storage improvements

There are many storage improvements included in vSphere 5.5. One that I am very happy about is the streamlined and scriptable UNMAP process used to identify and clear out free space on a VMFS datastore so that your storage array can re-thin and free the space on the back-end. This is a big one for me with 3PAR. I want to keep my LUNs on a diet and this new ESXCLI command will go a long way towards helping me do that. In addition to the new command line, the back-end functionality has changed with UNMAP, hopefully making it more acceptable to run during production hours and moving towards a time when automatic UNMAP during deletes might return to default functionality. UNMAP can now stagger the reclamation process so that it doesn’t saturate a backend array with IO.

Another nice feature, from an administrative standpoint, is the automatic removal of permanent device loss (PDL) state feature. This is especially handy in the event of array failures, which have been known to happen. In the event of a failure, automatically clearing a LUN in PDL will keep the administrator from having to perform rolling reboots to completely clean up the mess. The handling of PDL conditions has improved greatly over the versions I have used from 3.0 forward. There as a time that a PDL could completely hose an ESX host and a reboot was the only way to recover.

vSphere 5.5 also features enhancements around SSD and Flash storage. The ESXi hosts now have the ability to work with add-add and removal of SSD devices, allowing a graceful handling of SSD storage as it becomes increasingly more common. In the world of Flash, a new read cache feature is introduced allowing for the aggregation of Flash storage into a pool that can be consumed by VMs for the purpose of accelerating read intensive workloads. Last, but not least, 16Gb Fiber Channel support, end to end, is available in vSphere 5.5.

Network enhancements

In the networking world, vSphere Distributed Switches just got a needed boost of capabilities. Traffic filtering has been added allowing administrators to filter the types of traffic. QOS has been added to the mis allowing for prioritization of traffic from the vSwitch up to the physical switch layer. Security officers and admins troubleshooting will both be happy to see the addition of a host level packet capturing tool that is the equivalent of tcpdump to Linux admins. 40Gb NIC support is also being introduced with vSphere 5.5.

Mission critical workloads

One of the message points from VMware during VMworld 2013 was about making vSphere a platform that can run any mission critical workload. The maximum enhancements and other resiliency features are all targeted towards that goal, but in particular one additional capability was introduced with vSphere 5.5 that is meant to assist with virtualizing latency sensitive workloads, like VOIP. A new Latency Sensitivity setting is now available on VMs which help reduce latency for necessary virtual workloads. When set to a ‘high’ setting, the hypervisor actively tries to reduce latency by reserving memory, dedicating CPU cores and disabling network feature prone to latency.

Tying it all together…

There is really so much more in vSphere 5.5 – a couple great places to get more detail is available from Kyle Gleed in his A Summary of What’s New in vSphere 5.5 post and in the What’s New in vSphere 5.5 document on the VMware website.

All in all, I left the Monday morning keynote of VMworld 2013 feeling a little let down. Many of the enhancements that will really improve the admin’s life didn’t get any stage time or mention during the keynote – one reason for the delay in posting details – I had to find details… Two of the big deals that were given stage time this year were the VMware VSAN, a software defined virtual storage implementation baked heavily into vSphere, and NSX, the network virtualization platform that VMware acquired with Nicira. I hope to post on these two in depth in the coming weeks.

I was very happy to see a bit of a return by VMware to its core audience and products – centered around datacenter virtualization. I had the opportunity to talk to a lot of other bloggers and industry folks while I was at the conference and I found several who shared my view that VMware had strayed from its roots in years past chasing the cloudy dream — a dream that we felt was a bit beyond the masses. And while cloud adoption is growing, most enterprise datacenters are still reliant upon VMware and its vSphere platform to running daily operations. Its good to see that VMware did not abandon us all and that they are are still working very hard to build the best and most resilient datacenter platform available.

Questionable decisions in years past – like the nasty 4 letter word – VRAM – can now be jokes and chuckles. With so much attention given to vCloud and programming layers in the past couple years, it was welcoming to see things added to vSphere, where I live and work. In my opinion, this year VMware had to deliver something valuable to all of its customers to starve off competition and justify the continued spend for Service and Subscription (SnS) contracts. In many ways, I think they delivered. I’ll save my thoughts on where they still miss for another post.

September 9, 2013 0 comments

Datacenter Storage

vSphere 5.5 introduces new, simpler UNMAP command

by Philip Sellers August 26, 2013

written by Philip Sellers

Welcome news for anyone using array-based thin provisioning with vSphere, VMware has introduced a new and simpler UNMAP command that can be invoke from esxcli with vSphere Platform 5.5. This will help 3PAR customers more easily perform the UNMAP process to free space on LUNs so that the array can discover and free the space on the back-end. This is a big deal for any thin-provisioned LUNs in a vSphere environment. In addition to an easier way to invoke the UNMAP, VMware has also made changes in how the UNMAP occurs on the array.

According to VMware’s What’s New document, the previous method of creating a temporary file and then informs the array the the blocks in the temporary file can be reclaimed. In the case of the 3PAR, the zero detect process then runs to identify the free space and reclaim it. With version 5.5, the reclaim allows the administrator to specify the size in blocks rather than a percentage and now the dad space can be reclaimed in increments instead of all at once – to help shops that experience performance degradation with the old method.

The new command is run from esxcli, logically under storage and VMFS.

[code]esxcli storage vmfs unmap[/code]

The full syntax of the command is not clear yet and I’ll have to wait for my 5.5 upgrade to test or get more details.

The fact that the new UNMAP is run from esxcli makes me hopeful that I will be able to script and schedule the UNMAP to occur during non-peak hours when the maintenance would not be service affecting.

What is UNMAP and what does it do? See this post for more information.

August 26, 2013 1 comment