Tag:

Bladesystem

Stick to the recipe to keep ESXi on HP hardware cooking

by Philip Sellers November 6, 2014

Back in September, I blogged about a potential problem between HP’s September software bundle for ESXi and ProLiant Gen8 blade servers with Emulex CNA’s. Long story short, the issue was caused by a mismatch of firmware between Virtual Connect and the Emulex CNA’s. The mismatch was induced by me, as the administrator, not keeping all my components on a tested firmware ‘recipe’ from HP.

A couple years ago, HP made a shift in the BladeSystem firmware that unified the individual components into a tested configuration and began publishing the firmware together as release sets. This meant that if you installed a recipe of Onboard Administrator, Virtual Connect and iLO firmwares, they would be more stable since they had been tested together for interoperability. About a year later, HP unified software drivers and firmware on all ProLiant systems in a similiar way with the release of the Service Pack for Proliant (SPP). The Service Pack for ProLiant did the same thing for OS level drivers and hardware firmware that the earlier release sets did for BladeSystem firmware. All components were tested together at specific levels and the SPP’s unified release made it easy for administrators to apply a blessed recipe of software across components.

HP also does a similar thing with its OEM version of ESXi, which includes additional software components that allow a Common Information Model (CIM) implementation within the OS to display hardware status among other things. Because of the add-in software, HP maintains a recipe between this software and the SPP bundle and firmware levels required and tested on their hardware platforms.

The moral of the story is to stay with HP’s recipes and SPP releases. When you deviate from these, having firmware older than recommended along with newer drivers or agents, problems can occur. In my own experiences, when I have stayed with the recipe, I haven’t experienced major malfunctions. As with any software, new releases fix different bugs and there are times when you want to move outside of a recipe, if a newer version fixes a specific issue you’re experiencing, but in general, the rule is stay in recipe.

The other great benefit that you receive as a customer is a guarantee of support. Traditionally, HP support pointed to older firmware as the first course of action for many support calls. Following the SPP and ESXi recipe models, you now get a year of support from the release date of a bundle. With this, you can push back on support and say “I’m in a supported configuration, now support me.” if you’re ever asked to update firmware without troubleshooting first. This is a great position for the customer. Support can also point to specific resolutions in release notes if there is a problem resolved that requires you to move to newer release, but following HP’s guidelines ensures that all the separate components have been tested together for the best possible operational environment that you can have.

And that advice is based on years of struggle in the pre-release set world of HP hardware, experience with HP support and my own personal lapses in following my own advice… Hope this helps you avoid my struggles…

Service Pack for ProLiant – http://www.hp.com/go/spp

VIBS Depot from HP – http://vibsdepot.hp.com

November 6, 2014 2 comments

Compute Datacenter

HP moving from release sets to ‘Service Packs’ for BladeSystem

by Philip Sellers January 30, 2012

written by Philip Sellers

HP is moving away from the release sets that it introduced in 2010 to a unified Service Pack for Proliant (SPP) model for updating firmware and software on the HP BladeSystem along with all Proliant servers. I had previously reported about the software release sets back in June of 2010.

In the latest realignment, HP is consolidating these release sets with its Proliant Support Packs (PSP’s) and the Firmware releases that were previously three separate distributions. In addition, HP is adopting a date style of numbering, matching the release set scheme, rather than the version numbering previously used with PSP and Firmware releases. This will allow administrators to quickly recognize how behind a system is from the current release.

Consolidation also allows a single interface, qualification schedule and unified release for customers on all systems. Although a solid step, the release sets did not completely match the release of Firmware DVD’s from HP and sometimes there were issues with support where case workers asked that you update firmware out of compliance with a release set. During their first year of existence, release sets came out often and in many ways, shored up stability in our blade environment, but in the past year, they seemed to have lagged behind in releases.

Back when I first covered release sets, I also noted that it the release sets were a very good thing for customers, since the sets gave a supportable configuration and schedule of compliance. Although I didn’t say it, compliance meant that the customer could finally force the support side of HP to work a case instead of hiding behind the ‘please patch to latest firmware’ excuse. My hope is that the SPP will do the same in a unified firmware and driver software release.

On many of our critical systems, we have observed an increased interdependency between the firmware versions and the driver versions which need to be loaded together to gain stability. Consolidating all these separate releases into a single engine and distribution makes logical sense as these dependencies increase.

In an effort to target exactly what customers need, there will be a master distribution of the SPP along with 6 additional subset versions targeted towards specific needs, per the HP website. These will be smaller, which translates to faster downloads and a more customized installation, including only what is required for these solutions.

HP touts four major areas of improvement by introducing the SPP — it reduces customer qualification cycles, reduces resource usage, maintenance windows and downtime. Since firmware patching is typically not the most fun part of a system administrators job, reducing the frequency of these upgrades is a welcome change.

HP plans to offer 3 to 5 of these SPP’s per year. The first version released is version 2011.09.0 and 2012.01.0 is expected soon.

January 30, 2012 1 comment

Compute Datacenter Storage

Overview and experiences with HP BladeSystem.

by Philip Sellers August 13, 2011

written by Philip Sellers

For anyone who found this post looking for my week two post for the Blogger Reality Show, please see the Simplifying IT support and deployments with converged systems post instead. This post is just more or less general info about HP BladeSystem and my experiences.

I worked with HP to deploy two HP BladeSystem C7000 chassis in my work datacenters about three years ago. Our group has two enclosures in to separate datacenters which are mirrors of each other in configuration. At first, we only ran Windows clusters on the BladeSystems with one node in each enclosure. We considered this a safe way to get familiar with the technology, management and understand the reliability of the BladeSystem in our environment. We, as others have, did find the enclosures to be very reliable, eventually reconfiguring them to add Virtual Connect Flex-10 and moving some of our ESX onto BladeSystem. Today, HP BladeSystem runs a very large percentage of our overall infrastructure.

August 13, 2011 1 comment

Compute Datacenter

HP adds firmware release sets for Bladesystem

by Philip Sellers June 22, 2010

written by Philip Sellers

HP has updated their framework for BladeSystem firmware and driver releases through a consolidated Service Pack for Proliant. See this post for more information.

HP added the idea of release sets for Bladesystem firmware starting in January. On a conference call yesterday, we were alerted to the new release set certification process. Previously, HP had been releasing firmware for the Bladesystem as components and firmware were updated, which they still do, but the idea of a release set adds an additional cross-testing process to ensure that firmware from each component works together correctly. There was no publicly disclosed certification process prior to January.

To quote the HP engineer on the call — there were just too many combinations and possibilities to be able to certify all available firmwares — and so the idea of a release set began in January. The release sets are available in a compatibility matrix on the HP Bladesystem page (http://h18004.www1.hp.com/products/blades/components/c-class.html) — look at the Compatibility tab.

The good news is that in our environment, we are close to compliance with the January release set. The only thing out of compliance for the January release is our blade ROM, PMC and iLO firmware, but, as we were also informed, this is not a good situation to be in.

Something I already knew is that we have a bottom-up firmware topology for the HP Bladesystem — meaning that we have to update the bottom level components first and then move up. I knew this applied to the interconnects and onboard administrator modules, but it also extends to blade servers also. In addition, the blade servers are at the lowest level and should be updated first — particularly the iLO.

The reason for this is that new OA and interconnect firmware may introduce features and if the PMC or iLO is not aware of these features due to out-dated firmware, erroneous might be passed and all sorts of things could happen — at worst, random server reboots.

iLO firmware 1.81 is particularly susceptible to the random reboot for Windows blades. If you are running 1.81 and have Windows OS loaded, you should upgrade to 1.82 as soon as possible. See http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?locale=en_US&objectID=c01802766 for more information. Other OSes – Linux, VMware, XenServer – are not affected.

http://h18004.www1.hp.com/products/servers/service_packs/en/index.html

June 22, 2010 2 comments

Compute Datacenter

Upgrading Virtual Connect Ethernet to VC-Ethernet Flex-10 on an HP Bladesystem

by Philip Sellers November 23, 2009

written by Philip Sellers

We recently made the purchase to upgrade our two blade enclosures to HP’s Flex-10 Ethernet technology and last week, with the help of a partner, we performed the install and upgrade of the Virtual Connect domain.

HP Virtual Connect Flex-10 allows an administrator to take a physical 10G connection and partition it into up to 4 more appropriately sized logical NICs presented to a server blade. Virtual Connect Flex-10 incorporates all of the benefits of Virtual Connect (previously explained here) while allowing for additional logical NICs. A Flex-10 Ethernet interconnect module with two onboard Flex-10 NICs on a blade will allow for 8 total NICs – something formerly only available if all 8 interconnect bays and the two mezzanine slots were populated with Ethernet modules. This greatly reduces the amount of cabling required at the blade enclosure and condenses the amount of connections required.

The Flex-10 interconnect modules include SFP connectors which are capable of copper or fiber connections depending on the GBIC installed.

Upgrade Process
We expected going into the installation that we could need to recreate the Virtual Connect domain (a domain is the configuration, server profiles and settings for a Virtual Connect deployment on a specific enclosure or set of enclosures). Since we only had 5 server blades installed and operating, we weren’t overly concerned. Our primary concerns were to ensure that the same Virtual Connect MAC and WWID settings were reinstalled and that our recreated server profiles matched their MAC address and WWID settings from the prior installation. So, to ensure we had all the information, we made printouts of the server profiles, Ethernet networks and fiber channel configuration and then proceeded to installing the modules.

We shutdown all of the existing blades from service. We removed the existing VC-Ethernet modules from Bays 1 and 2 and we installed the new VC-Ethernet Flex-10 modules in these bays. The VC-Eth modules in Bays 1 and 2 contain the domain configuration. Once removed, the domain had to be completely recreated. The new modules were at a higher firmware (2.12) versus the original modules (2.01).

What surprised us during installation was that we were able to successfully restore our original Virtual Connect domain configuration onto the Flex-10 modules. Once restored, we were able to see all configuration and the only thing showing an error were the Ethernet networks and shared uplink sets. These didn’t have any active ports assigned to the uplinks. The new VC-Eth Flex 10 modules have a different port numbering to differentiate from the original VC-Eth modules. After reassigning ports to the uplink sets, all of the Ethernet networks appeared to be back online. The firmware differences didn’t appear to be a problem after initial restore.

After booting servers onto the enclosure (all are boot-from-SAN and booted fine), we determined that we were having some network connectivity issues. After troubleshooting, we upgraded VC firmware on the modules. This proceeded without problems, but upon reboot, the VC-Ethernet Flex-10 modules would not come back online. After troubleshooting, we installed upgraded firmware on the enclosure. The newer firmware on all seemed to resolve all issues and the enclosure was back online. Final time to upgrade was around 5 hours, but much of that was waiting on Virtual Connect modules to reboot and wondering why they wouldn’t come back online.

Our second enclosure was expected to be a simple slam dunk after our largely successful and simple upgrade procedure — especially since we knew about firmware before on this round, but unexpected problems were there. This enclosure had a mid-plane replacement early in its life and apparently there was confusion with the Virtual Connect domain and the enclosure serial number. We received errors during restore and even after we forced the restore and upgraded firmware, the Ethernet networks never came back and responded.

This forced us into a recreate scenario. After recreating this VC domain by hand, everything came online and worked as wanted. I still believe that this is the cleaner way to handle the upgrade, although the restore worked on the first enclosure and presented very few problems.

Next… Configuring Flex-10 FlexNICs

November 23, 2009 2 comments