Tag:

VSAN

VMware VSAN ESA Re-balance

by Roger Lund October 15, 2023

written by Roger Lund

After I moved my VM’s to the vsan, I saw this warning.

So lets go ahead and enable auto re-balancing,

Lets go to the cluster, Configure, vSAN, And Services.

A screenshot of a computer

Description automatically generated with medium confidence

Lets click on edit, Under Advanced Options

A screenshot of a computer

Description automatically generated with medium confidence

Lets change automatic re-balance to enable, and click Apply.

A screenshot of a screen

Description automatically generated with low confidence

Now lets go to monitor, vSAN and Re-syncing Objects

A screenshot of a computer

Description automatically generated with medium confidence

As you can see, there are items with the intent to re-balance on the right.

And now there are none.

A screenshot of a computer

Description automatically generated with medium confidence

I asked Chat GPT the following to see what it thought about why we need to do this.

me: “in VMware vSAN, why is re-balancing important?”

ChatGPT

VMware vSAN (Virtual SAN) is a software-defined storage solution that pools local storage resources of ESXi hosts to provide a highly available and resilient shared storage solution for virtual machines. Re-balancing in vSAN is an essential operation for various reasons:

Even Distribution of Data: Over time, as workloads change, new data is written, and old data is deleted, the distribution of data across the disks and hosts can become uneven. Re-balancing ensures that the data is uniformly spread across the cluster, maximizing the utilization of resources.
Performance Optimization: An imbalanced cluster can lead to some disks or hosts being more heavily utilized than others. This can result in performance hotspots. By re-balancing the data, the I/O load is spread more evenly across the cluster, which can enhance overall performance.

Capacity Management: If one disk or host becomes fuller than others, it might run out of space even when other parts of the vSAN cluster have ample free space. Re-balancing helps in ensuring that storage capacity is used effectively, and no node runs out of space prematurely.
After Adding or Removing a Disk/Host: When you add new hosts or disks to a vSAN cluster, they start empty. vSAN’s re-balancing operation ensures that the new resources are utilized efficiently. Similarly, when a disk or host is removed, the data from that resource needs to be redistributed to maintain the desired fault tolerance levels.
Ensuring Compliance with Storage Policies: vSAN allows you to define storage policies that determine factors like fault tolerance and stripe width. If the data isn’t balanced properly, it might not meet the requirements of these policies. Re-balancing helps in ensuring that the data complies with the set policies.

Maintenance Operations: Before performing maintenance tasks or updates on a host, you may need to evacuate the data. After completing the task and bringing the host back, re-balancing can help restore the data distribution to its optimal state.
Health and Monitoring: vSAN health checks monitor for imbalances in the cluster. Re-balancing helps in rectifying those imbalances, thus ensuring the cluster remains in a healthy state.

It’s worth noting that VMware vSAN has automated mechanisms to handle some level of re-balancing operations in the background. However, there are scenarios, especially in rapidly changing environments, where manual intervention or closer monitoring might be necessary to ensure optimal performance and capacity utilization.

what do you think, is it far off?

That’s all for today 😊

October 15, 2023 0 comment

vExpert VMware VMware VSAN ESA vsan

VMware VSAN ESA with Intel Optane 900P drives.

by Roger Lund October 14, 2023

written by Roger Lund

Intel was kind enough to supply me with Intel Optane 900P drives for my Lab. As you know I am part of the VMware vExpert program, that allows things like this.

So naturally, the first thing I want to do is configure the VSAN ESA Express Storage Architecture. https://core.vmware.com/vsan-esa

Since AI is on our minds, I thought I would ask it to give us a overview and do a intro. Of course I used our favorite Chat GTP4.

This was done with VMware vSphere 8 update 1.

“

Greetings, dear readers! As an artificial intelligence developed by OpenAI, known as ChatGPT, I’m here to introduce you to a fascinating subject that’s changing the game in data storage and management: VMware’s Express Storage Architecture (ESA), especially in the context of the recently released vSAN 8 U1. This article has been commissioned to provide you with an understanding of ESA and to walk you through setting it up with Intel Optane PCIE 900P’s in Cisco UCS Hosts.

VMware’s vSAN 8 U1 marks a significant advancement in data storage with its introduction of ESA. This development enhances dis-aggregated storage, a robust and successful method for customers to share vSAN storage resources with other vSAN clusters or compute-only clusters. With the incorporation of ESA, users can now mount remote vSAN datastores housed in different vSAN clusters, leveraging an ESA cluster as an external storage resource for a vSphere cluster. All this while maintaining the ease of use that customers have come to appreciate from vSAN’s Original Storage Architecture (OSA).

Another salient feature of vSAN 8 U1 is the support for vSAN stretched clusters with disaggregation in vSAN, made possible by the OSA. This feature optimizes network paths for specific topologies, leading to improved performance capabilities in stretched cluster configurations.

Lastly, vSAN 8 U1 allows for dis-aggregation across clusters managed by different vCenter server instances, offering a level of scalability and flexibility previously unattainable.

However, this high-level overview only scratches the surface of what VMware’s Express Storage Architecture truly brings to the table. The meat of our exploration lies in how we can set up ESA using Intel Optane PCIE 900P’s in Cisco UCS Hosts, and that’s what we’ll focus on in the upcoming parts of this article.

So stay tuned, and together we’ll deep dive into the nuts and bolts of this innovative storage solution, exploring the possibilities and benefits it can bring to your organization’s IT infrastructure.
“

I then asked the following.

“now give a overview of how this ESA behaves differently than vSAN in lets say vsphere 7.x in this scenario. 3 UCS C220’s with 2 Optane drives each.”

“
VMware’s Express Storage Architecture (ESA) introduced in vSAN 8 U1 brings notable changes in the way vSAN manages and operates storage, especially compared to previous vSAN versions like those integrated into vSphere 7.x. For the purposes of this overview, let’s consider a scenario where we have three Cisco UCS C220 hosts, each equipped with two Intel Optane drives.

In the traditional vSAN environment you might have encountered in vSphere 7.x, storage resources are pooled together across the cluster, with each host contributing storage to the vSAN datastore. vSAN then handles the distribution and management of this storage resource, providing the illusion of a single, unified datastore. This distributed nature of vSAN allows for high resilience, as data is typically mirrored or erasure-coded across multiple hosts. However, this traditional architecture does not readily support the sharing of vSAN storage resources with other vSAN clusters or compute-only clusters.

Enter VMware’s Express Storage Architecture in vSAN 8 U1. ESA introduces the capability of storage dis-aggregation, allowing the vSAN storage resources to be shared with other vSAN clusters or even with compute-only clusters. So in our scenario, if one of the UCS C220’s with its pair of Optane drives was part of a separate vSAN cluster (or was a compute-only cluster), it could still access and use the storage resources of the other two hosts via ESA. This results in enhanced flexibility and utilization of storage resources.

ESA also supports dis-aggregation in vSAN in both standard and stretched cluster configurations, although some limitations exist. While it brings added capabilities, there are features such as cross-vCenter Server capabilities, support for stretched clusters, and de-duplication that are not currently supported in ESA but are part of the traditional vSAN architecture.

The integration of Intel Optane drives plays a crucial role in both these architectures. In vSAN 7.x, Optane drives can be used as cache to dramatically improve the performance of the vSAN cluster. In vSAN 8 U1 with ESA, the same drives can still be used for caching, but they can also be used more flexibly because of the dis-aggregation feature. This means the performance benefits of the Optane drives can be enjoyed by all clusters with access to the dis-aggregated storage.

In conclusion, while the traditional vSAN as found in vSphere 7.x provides a solid and robust storage platform, the introduction of ESA in vSAN 8 U1 brings a level of flexibility and interoperability that was not previously possible, opening up new ways to maximize your infrastructure’s efficiency and performance.

“

Next I go to the datastore, and change the storage policy

Why? Lets see what Chat GPT says about it.

I asked “I set the storage policy to ESA Default Raid 5, explain how it differences from vSAN of old.”

“
The Express Storage Architecture (ESA) in VMware vSAN 8 has introduced significant changes to how RAID-5 is implemented and managed compared to previous versions of vSAN. This new architecture aims to provide improved performance, space efficiency, and adaptability. Here are the key differences when you set the storage policy to ESA Default Raid 5 compared to the traditional vSAN:

Performance and Space Efficiency: In previous versions of vSAN, using RAID-5 resulted in some performance trade-offs due to the computational complexity of erasure coding, despite its better space efficiency compared to RAID-1. With vSAN 8’s ESA, RAID-5 can now deliver the same level of performance as RAID-1, meaning you get RAID-1-like performance with the space efficiency of RAID-5. This is made possible by ESA’s log-structured file system and optimized data structure.

Erasure Coding Schemes: The traditional vSAN used a 3+1 RAID-5 erasure coding scheme, where data was written as a stripe consisting of 3 data bits and 1 parity bit, across at least 4 hosts. The ESA in vSAN 8 replaces this with two new RAID-5 erasure codes – a 4+1 scheme (4 data bits and 1 parity bit across at least 5 hosts) and a 2+1 scheme (2 data bits and 1 parity bit across at least 3 hosts). These new schemes offer better space efficiency, especially for smaller clusters.

Adaptive RAID-5: vSAN 8’s ESA introduces an adaptive RAID-5 feature that automatically adjusts the RAID-5 erasure code based on the host count in the cluster. This means the system will optimize the RAID-5 scheme for your current configuration without requiring manual intervention. This is a key improvement in terms of ease of management and ensuring optimal data resilience.

Data Resilience: The ESA Default Raid 5 provides better resilience in face of host failure. When a host fails, the system automatically adjusts to maintain data availability while also keeping the prescribed level of resilience.

Overall, these differences mean that when you set the storage policy to ESA Default Raid 5 in vSAN 8, you’re using a much more space-efficient, resilient, and performance-optimized version of RAID-5 compared to previous versions of vSAN. This all translates into a higher effective storage capacity, better performance, and simpler management for your vSAN clusters.

“

Let’s go ahead and set it!

A screenshot of a computer Description automatically generated with medium confidence

If we go to the VSAN services, we can see it now has the policy.

A screenshot of a computer Description automatically generated with medium confidence

Next we are going to migrate a VM to the workload

A screenshot of a computer Description automatically generated with medium confidence

Under VM Storage Policy, select the policy, and the vsanDatastore.

A screenshot of a computer Description automatically generated with medium confidence

A screenshot of a computer Description automatically generated

Click Finish

Lets go ahead and Migrate a VM to it.

A screenshot of a computer Description automatically generated

To show more movement, I queued another VM to migrate.

Ok, lets click on the cluster, monitor, vSAN and then performance, and click backend.

A screenshot of a computer Description automatically generated with low confidence

We can see some traffic.

Lets change to real time. Under the drop down.

A screenshot of a computer Description automatically generated with low confidence

To note, I only have a single 10GB link per host, until I can run some more cabling. And I am migrating from a Hybrid Truenas array.

More to come on this front for Sure!

October 14, 2023 0 comment

Datacenter Home Lab Storage Virtualization

Setting up VSAN in vSphere 6.5 > Hint, its super easy!

by Philip Sellers April 4, 2017

written by Philip Sellers

New job, new lab. I’ve been rebuilding our lab environment and for the first time, I have enough hardware to really give VSAN a whirl. I have 6 hosts with Intel Xeon E5645 procs and 4 x 400GB SSD’s in each server.

Being a lab environment, I knew I wanted to deploy a different SDS solution. I’ve worked with HPE’s StoreVirtual in the past and I was able to do a hands-on POC with an EMC ScaleIO SDS earlier this year. VSAN, however, had been elusive for me, so I took this opportunity to work through that solution.

The biggest hurdles in building VSAN is the underlying hardware. The software, as with most of VMware’s solutions, is dead simple to configure and make the solution work. Even though I have good hardware, I’m not strictly compatible per the HCL, but this doesn’t keep VSAN from working very well.

But my SSD’s show up as HDD’s

Back to the hardware – these older IBM System x’s – did not support JBOD passthrough on the RAID controllers. While it was a pain to setup, there is a workable solution. In the RAID controller, you have to build a RAID disk in a RAID 0 configuration for each drive and then present it out as a virtual drive. This is far from ideal, but works in a lab. The one problem it creates is masking the SSD capabilities from ESXi. ESXi sees these virtual disks as hard drives instead of solid state.

Fortunately, a little ESXCLI magic and you can override this – first you have to list all the local disks – the exact command will vary a little based on your hardware, but since all these were connected to my IBM servers, I was able to use a grep for IBM.

<code>esxcli storage core device list | grep IBM</code>

You just need to figure out what identifies your local disk type and then grep for the same thing.

For the rest, I followed the instructions in this VMware tech document – https://pubs.vmware.com/vsphere-55/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc%2FGUID-99BB81AC-5342-45E5-BF67-8D43647FAD31.html (these instructions work with 6.0 and 6.5, even though the document is for 5.5).

<code>esxcli storage nmp satp rule add -s SATP –device device_name –option=”enable_ssd”</code>

Lastly, you need the system to ‘reclaim’ the disk with the new policy.

<code>esxcli storage core claiming reclaim –device device_name</code>

Now the easy part

VSAN is built-in to vSphere 6.0 and 6.5, so via the vSphere Web Client, so setup is very simple. Instead of step-by-step, these instructions are the outline of steps to setup your own VSAN.

On each of your nodes to be used for VSAN, setup 10Gig VMKernel adapters and check the box for Enable Virtual SAN. You need at least one and I decided to re-use my iSCSI connections for the lab. [Someone familiar with best practices, comment is this is bad idea].
In the vSphere Web Client, navigate to your cluster object, navigate to the Configure tab for it and then select General under Virtual SAN.
Enable virtual SAN > click Edit… next to Virtual SAN and then check the box to “Turn ON Virtual SAN.” [as a side note, I’ve always loved how easy HA clustering is to enable, and Virtual SAN is just as simple – although you have a little config left to complete]

I kept the default of adding disks Manually to my installation. If you have JBOD passthrough and your SSD’s show up as SSD’s automatically, go for Automatic disk add…

Next step is adding disks, just a simple – go to Disk Management, click the Claim Disks button and then for each host, choose how you’d like to adopt the disks. For my systems, I chose Capacity at the host level, which is inherited to all disks in the host, then I went back and chose one Cache disk per host. Click OK and your VSAN is building.

You’ll see a number of tasks in the background under Recent Tasks.

Last, you’ll see a new Datastore presented from your VSAN which is an aggregate of all the disks and groups from all your participating systems.

One last tip, go ahead to the Monitor tab and then go to Performance. Click on Virtual SAN – either option – and then enable performance monitoring.

Why VSAN for the lab?

I had several reasons…

No drive penalty… With solutions like HPE’s StoreVirtual and EMC’s ScaleIO, you must have a VM ‘controller’ that is the main connection to your local storage and presents it up to servers for consumption. StoreVirtual has a virtual appliance and ScaleIO has its SDS in virtual appliance form – but with a VM, you have to store the base VM image somewhere, meaning one less drive to be setup in your drive pool.

It is tightly integrated into vSphere, meaning less management points.
I have entitlements for my lab via the vExpert program, so price is not an issue. For others, check out VMUG Advantage – while not free, it is a great value for your home lab.

April 4, 2017 0 comment

Datacenter News Storage VMware vSphere

VMware Virtual SAN 6.0

by Roger Lund February 11, 2015

written by Roger Lund

The following Blog post is about VMware VSAN or Virtual San. http://www.vmware.com/products/virtual-san

The following post was made with pre GA , Beta content.

All performance numbers are subject to final benchmarking results. Please refer to guidance published at GA

All Content and media is from VMWare, as part of the blogger program.

Please also read vSphere 6 – Clarifying the misinformation http://blogs.vmware.com/vsphere/2015/02/vsphere-6-clarifying-misinformation.html

Here is the published What’s New: VMware Virtual SAN 6.0 http://www.vmware.com/files/pdf/products/vsan/VMware_Virtual_SAN_Whats_New.pdf

Here is the published VMware Virtual SAN 6.0 http://www.vmware.com/files/pdf/products/vsan/VMware_Virtual_SAN_Datasheet.pdf

Whats New?

Disk Format

New On-Disk Format
New delta-disk type vsanSparse
Performance Based snapshots and clones

VSAN 5.5 to 6.0

In-Place modular rolling upgrade
Seamless In-place Upgrade

Seamless Upgrade Rollback Supported
Upgrade performed from RVC CLI
PowerCLI integration for automation and management

Disk Serviceability Functions

Ability to manage flash-based and magnetic devices.

Storage consumption models for policy definition
Default Storage Policies
Resync Status dashboard in UI

VM capacity consumption per VMDK
Disk/Disk group evacuation

VSAN Platform

New Caching Architecture for all-flash VSAN
Virtual SAN Health Services
Proactive Rebalance

Fault domains support
High Density Storage Systems with Direct Attached Storage
File Services via 3rd party

Limited support hardware encryption and checksum

Virtual SAN Performance and Scale Improvements

2x VMs per host

Larger Consolidation Ratios
Due to increase of supported components per hosts
9000 Components per Host

62TB Virtual Disks

Greater capacity allocations per VMDK

VMDK >2TB are supported

Snapshots and Clone

Larger supported capacity of snapshots and clones per VMs
32 per Virtual Machine

Host Scalability

Cluster support raised to match vSphere
Up to 64 nodes per cluster in vSphere

VSAN can scale up to 64 nodes

Enterprise-Class Scale and Performance

vsan02

VMware Virtual SAN : All-Flash

Flash-based devices used for caching as well as persistence

Cost-effective all-flash 2-tier model:oCache is 100% write: using write-intensive, higher grade flash-based devices
Persistent storage: can leverage lower cost read-intensive flash-based devices

Very high IOPS: up to 100K(1)IOPS/Host
Consistent performance with sub-millisecond latencies

30K IOPS/Host100K IOPS/Hostpredictable sub-millisecond latency

Virtual SAN FlashCaching Architectures

All-FlashCache Tier Sizing

Cache tier should have 10% of the anticipated consumed storage capacity

Measurement Requirements	Values
Projected VM space usage	20GB
Projected number of VMs	1000
Total projected space consumption per VM	20GB x 1000 = 20,000 GB = 20 TB
Target flash cache capacity percentage	10%
Total flash cache capacity required	20TB x .10 = 2 TB

Cache is entirely write-buffer in all-flash architecture
Cache devices should be high write endurance models: Choose 2+ TBW/day or 3650+/5 year
Total cache capacity percentage should be based on use case requirements.

–For general recommendation visit the VMware Compatibility Guide.

–For write-intensive workloads a higher amount should be configured.

–Increase cache size if expecting heavy use of snapshots

New On-Disk Format

Virtual SAN 6.0 introduces a new on-disk format.
The new on-disk format enables:

–Higher performance characteristics

–Efficient and scalable high performance snapshots and clones

–Online migration to new (RVC only)

The object store will continue to mount the volumes from all hosts in a cluster and presents them as a single shared datastore.

The upgrade to the new on-disk format is optional; the on-disk format for Virtual SAN 5.5 will continue to be supported

Performance Snapshots and Clones

Virtual SAN 6.0 new on-disk format introduces a new VMDK type

–Virtual SAN 5.5 snapshots were based on vmfsSparse (redo logs)

vsanSparse based snapshots are expected to deliver performance comparable to native SAN snapshots.

–vsanSparse takes advantage of the new on-disk format writing and extended caching capabilities to deliver efficient performance.

All disks in a vsanSparse disk-chain need to be vsanSparse (except base disk).

–Cannot create linked clones of a VM with vsanSparse snapshots on a non-vsan datastore.

–If a VM has existing redo log based snapshots, it will continue to get redo log based snapshots until the user consolidates and deletes all current snapshots.

Hardware Requirements

Flash Based Devices In Virtual SAN hybrid ALL read and write operations always go directly to the Flash tier.
Flash based devices serve two purposes in Virtual SAN hybrid architecture

1.Non-volatile Write Buffer (30%)
–Writes are acknowledged when they enter prepare stage on the flash-based devices.
–Reduces latency for writes2.

2. Read Cache (70%)
– Cache hits reduces read latency
– Cache miss – retrieve data from the magnetic devices

Flash Based Devices In Virtual SAN all-flash read and write operations always go directly to the Flash devices.

Flash based devices serve two purposes in Virtual SAN All Flash:

1 .Cache Tier
–High endurance flash devices.
–Listed on VCG2.
2. Capacity Tier
–Low endurance flash devices
–Listed on VCG
Network

•1Gb / 10Gb supported
–10Gb shared with NIOC for QoS will support most environments
–If 1GB then recommend dedicated links for Virtual SAN
–Layer 3network configuration supported in 6.0

•Jumbo Frames will provide nominal performance increase
–Enable for greenfield deployments
–Enable in large deployments to reduce CPU overhead

•Virtual SAN supports both VSS & VDS–NetIOC requires VDS
•Network bandwidth performance has more impact on host evacuation, rebuild times than on workload performance

High Density Direct Attached Storage

–Manage disks in enclosures – helps enable blade environment
–Flash acceleration provided on the server or in thesubsystem
–Data services delivered via the VSAN Data Services and platform capabilities
–Direct attached and disks (flash devices, and magnetic devices) are Supports combination of direct attached disks and high density attached disks (SSDs and HDDs) per disk group.

Users are expected to configure the HDDAS switch such that each disk is only seen by one host.

–VSAN protects against misconfigured HDDASs (a disk is seen by more than 1 host).
–The owner of a disk group can be explicitly changedby unmounting and restamping the disk group from the new owner.

•If a host who own a disk group crashes, manual re-stamping can be done on another host

.–Supported HDDASs will be tightly controlled by the HCL (exact list TBD).•Applies to HDDASs and controllers

FOLLOW the VMware HCL. www.vmware.com/go/virtualsan-hcl

Upgrade

Virtual SAN 6.0 has a new on disk format for disk groups and exposes a new delta-disk type, so upgrading from 1.0 to 2.0 involves more than upgrading the ESX/VC software.

Upgrades are performed in multiple phases

Phase 1: Fresh deployment or upgrade of to vSphere 6.0

vCenter Server

ESXi Hypervisor

Phase 2: Disk format conversion (DFC)

Reformat disk groups

Object upgrade

Disk/Disk Group Evacuation

In Virtual SAN 5.5 in order to remove a disk/disk group without data lost, hosts were placed in maintenance mode with the full data evacuation mode from all disk/disk groups.
Virtual SAN 6.0 Introduces the support and ability to evacuate data from individual disk/disk groups before removing a disk/disk group from the Virtual SAN.

Supported in the UI, esxcli and RVC.

Check box in the “Remove disk/disk group” UI screen

Disks Serviceability

Virtual SAN 6.0 introduces a new disk serviceability feature to easily map the location of magnetic disks and flash based devices from the vSphere Web Client.

Light LED on failures
When a disk hits a permanent error, it can be challenging to find where that disk sits in the chassis to find and replace it.
When SSD or MD encounters a permanent error, VSAN automatically turns the disk LED on.
Turn disk LED on/off
User might need to locate a disk so VSAN supports manually turning a SSD or MD LED on/off.
Marking a disk as SSD
Some SSDs might not be recognized as SSDs by ESX.
Disks can be tagged/untagged as SSDs
Marking a disk as local
Some SSDs/MDs might not be recognized by ESX as local disks.
Disks can be tagged/untagged as local disks.

Virtual SAN Usability Improvements

What-if-APIs (Scenarios)
Adding functionality to visualize Virtual SAN datastore resource utilization when a VM Storage Policy is created or edited.

–Creating Policies

–Reapplying a Policy

Default Storage Policies

A Virtual SAN Default Profile is automatically created in SPBM when VSAN is enabled on a cluster.

–Default Profiles are utilized by any VM created without an explicit SPBM profile assigned.

vSphere admins to designate a preferred VM Storage Policy as the preferred default policy for the Virtual SAN cluster

vCenter can manage multiple vsanDatastores with different sets of requirements.
Each vsanDatastore can have a different default profile assigned.

Virtual Machine Usability Improvements

Virtual SAN 6.0 adds functionality to visualize Virtual SAN datastore resource utilization when a VM Storage Policy is created or edited.
Virtual SAN’s free disk space is raw capacity.

–With replication, actual usable space is lesser.

New UI shows real usage on

–Flash Devices

–Magnetic Disks

Displayed in the vSphere Web Client and RVC

Virtual Machine >2TB VMDKs

In VSAN 5.5, the max size of a VMDK was limited to 2TB.

–Max size of a VSAN component is 255GB.

–Max number of stripes per object was 12.

In VSAN 6.0 the limit has been increased to allow VMDK up to 62TB.

–Objects are still striped at 255GB.

62TB limit is the same as VMFS and NFS so VMDK can be

There it is. I tried to lay it out as best I could.

Want to try it? Try the Virtual SAN Hosted Evaluation https://my.vmware.com/web/vmware/evalcenter?p=vsan6-hol

What’s New in Virtual SAN 6

Learn how to deploy, configure, and manage VMware’s latest hypervisor-converged storage solution.

HOL-SDC-1408-testdrive .

– See more at: https://my.vmware.com/web/vmware/evalcenter?p=vsan6-hol#sthash.OcuO1gXQ.dpuf

Roger Lund

February 11, 2015 1 comment

Datacenter Storage Virtualization

VMware releases VSAN and the power of policies

by Philip Sellers September 19, 2013

written by Philip Sellers

VMware Virtual SAN (VSAN)
Diagram by VMware

VMware is pushing back on storage vendors as it seeks to show the world the full power of software-defined storage. A couple weeks ago, VMware announced the VMware Virtual SAN (VSAN), a new software-defined storage platform tightly coupled with vCenter server. The new platform requires both hard disks and solid state disks to create a pool of redundant, shared storage within a cluster. The hard disks are the core of the storage and the SSD are used for buffering and read caching but do not contribute to the total amount of storage. And although the name has SAN in it, the traditional SAN that is bread and butter of its parent company, EMC, could not be more different than VSAN.

In many ways, this is similar to the VMware Virtual Storage Appliance, but VSAN is much more integrated into vCenter and the vSphere platform. The requirement of SSD is also new and will greatly contribute to overall responsiveness for the storage. But more importantly, the new software defined storage platform is an example of the possibilities yet to come from tightly integrating storage with vSphere.

The real power of what VMware has released is not the storage technology itself or the simplicity that it can be deployed (single-click simplicity like HA and DRS). The real power is the addition of Storage Policies completely throughout vSphere and VSAN. The policies are the real capability that can be leveraged across multiple types of storage. As I said earlier, I think VMware is simply showing all storage vendors this is what is possible. The VSAN is certainly not a SAN replacement for most enterprise customers, but there are certainly remote office or small office use cases where this will fit nicely without the need of purchasing a purpose built storage array — use cases, by the way, where virtual storage appliances (VSA) currently fit.

But back to the policies. Introduced in vSphere 5.0, Storage Policies allow administrators to tag and set classing of storage for different pools that are available within a vSphere cluster. The Storage Policies allow an administrator to abstract the back-end storage and provide a list of capabilities to end-users or operators who are creating new virtual machines – and choose a policy during initial deployment to meet their needs. You see, what VMware has discovered is that the end users don’t care if the back-end storage is a 3PAR or a VMAX – a Synology or a Drobo. What they care about is a class of service provided, response times and how those translate to their application and its performance. They also care about the cost of charge-back, if an organization has implemented those types of billing scenarios. Policies are all a way of defining those attributes back to the end user and abstracting the actual storage from those decisions.

I am sure many VMware customers are like myself. When I started the virtualization project, most systems ran on local storage. The SAN was reserved for our most mission critical systems and most of those were clustered. Today, however, vSphere has moved all of those workloads onto the SAN. It has increased my resiliency and ability to survive a disaster, but its also driven storage costs up exponentially. Yes, VMware is owned by EMC – goal to sell disks accomplished.

But the VSAN must run a bit counter to the EMC culture, so back to my original point. In many ways, I think the VSAN is simply a proof of concept of what is possible with software defined storage. It is a way for VMware to show the rest of the world what we can do. VMware is very good at fostering its ecosystem and partner vendors, so I think this will spur new innovation.

And so, for a third time, back to policies. With the introduction of VSAN, VMware is extending this capability of Storage Policies to allow administrators to assign a policy to a VM on creation and then the policy lives with the VM allowing policy based management of the VM over time. Not only can it be setup initially with the VM, it can follow and adapt as the back-end storage changes over time. With the addition of storage pools and maintenance mode for storage, VMware has automated a lot of the manual labor around managing storage. By creating the pools of storage and moving datastores in and out of them, administrators and move or remove storage without interrupting service of virtual machines. And also by applying tags and policy around those storage pools, it allows for the addition and removal of datastores as required to onramp or offramp storage as purchasing or lease cycles dictate. All of this while maintaining service levels and agreements for customers and workloads – whether those customers are internal or external to your organization.

Beyond backend maintenance of storage, say the attributes of a particular application or VM changes over time. Say it now requires a higher tier of storage – well thats an easy policy change as well on the VM and with the magic of storage vMotion and fully-automated storage DRS, the vSphere platform will perform the necessary moves to increase performance and move the workload onto the appropriate storage de jour.

In many ways, this is all done with the idea of reducing the menial tasks that an administrator has to perform. Its mean to kill the tracking spreadsheets and other mechanisms we have traditionally relied upon for classing storage or moving workloads appropriately. It all goes to making the storage layer of vSphere smarter and more automated.

Now, the last piece of policy that really sets VSAN apart from the competition. Within VSAN, the vSphere administrator can dictate the type of RAID, the number of stripes and copies that are present for a particular VM or datastore. For the first time, the core vSphere platform is controlling storage down the stack and automating how it operates. vSphere policy now defines where and how replication occurs, how and where stripes of data are implemented, and on a per VM basis, quality of service attributes to prevent noisy neighbor problems. And combining those together gives context and visibility up the stack in vSphere, too. The two way communication of the policy and the storage implementation means that the administrator can more granularly dictate how the storage should act to support the requirements of the virtual workload.

So, I have to say, after a week in San Francisco at VMworld 2013, the VSAN is the one piece I am most excited to test and evaluate. If for no other reason, you should sign up for the VSAN beta to see where the future of storage is going. I think the writing is on the wall of every vendor is talking software-defined, the VSAN represents the first software-defined storage product with such a deep level of integration into vSphere. I fully expect to see other vendors like Nexenta and HP StoreVirtual quickly adopting this paradigm – maybe even making it to market with product offerings around the same time that VSAN becomes generally available. API calls and other integration points should be very simple for software storage implementations. I’m excited to see where this next step takes us.

September 19, 2013 0 comment