Category:

Trends

Data proximity is driving analytics to the cloud

by Philip Sellers January 13, 2016

written by Philip Sellers

shutterstock_149459630 Unique, new types of datasets are forcing companies to investigate cloud-powered analytics to tap into data that can steer a company’s direction and its decision making. These datasets have challenging characteristics that require moving the analytics platform and not the data that should be mined.

A few months ago, I was part of a briefing from Teradata, one of the largest data warehouse and analytics software vendors in the world. Teradata announced that they would be bringing their analytics platform to Amazon Web Services and offering their data platform on a utility based consumption model. But it wasn’t the announcement as much as the reason that I found really interesting on the call. According to Teradata, data proximity is one of the key driving factors to move analytics onto cloud platforms like Amazon’s.

Big Data has been an industry buzzword for several years, now. Working for a medium business, its occurred to me that big data and big data problems just were outside of my area of concern. We don’t have that much data, I thought, but I have realized I was wrong. You don’t have to be a big business to have big data problems and benefit from the analysis of big data sets. The rate at which data is being generated has exploded in the last couple years and it will continue to explode in years to come because of new innovation.

New datasets

First, there are massive amounts of data being generated online through social networks like Facebook, Twitter, Instagram and others. This data has the potential to provide insights into customer perception and concerns for any brand and company, if they can tap into the data and make sense of it.

Sensor data is another major area of concern. Enterprise has long had appliances with sensors that report back environmental conditions in data centers. Hardware vendors have built sensors into their servers and racks to report back hot-spots and cold-spots and look at optimal power usage and optimize operational costs. But the truth is that Internet of Things is bringing a massive expansion of sensor data. Everything from your toaster to light bulbs have sensors and data that can be tapped for different uses. Depending on the business and use cases, there are massive datasets created by these devices that may have use cases for the enterprise.

As another example, municipalities are increasingly deploying cameras throughout their territory to watch for accidents and gain visibility when crimes occur. Video takes an immense amount of storage and event more than just storing the data, making sense of that data once it is acquired is more than a human and manually perform. This requires an automated software method to analyze and make sense of the video data being transmitted.

A common trait

All three of these examples share one common trait. The data generated exists outside of the corporate firewall – the usual place where analytics platforms are run – and sometimes outside of the control of the enterprise – as is the case with Internet sites and sensor data from products sold to customers. This presents the idea of data proximity and locating the analytics platform closer to the data it should mine.

Data proximity is an interesting concept. Cloud services – like Twitter, Facebook and other social services – have created a ton of new data but it is not necessarily data that needs to be transported and stored within the confines of the corporate firewall. Datasets like video are far too large to be transported once they are at rest and this may also be a driving factor for locating the analytics closer to the dataset.

Teradata’s solution

The Teradata Database on AWS is the solution announced in October. Running on the Amazon Web Services, it offers customers a pay-as-you-go consumption model with self-provisioning services and scale up capabilities. It joins Teradata’s existing Teradata Cloud offering that is a dedicated analytics platform hosted by Teradata in its datacenters. While the Teradata Cloud is targeted at high-performance analytics, the Teradata Database on AWS is targeted at middle ground customers who need a reliable, production analytics platform with the economies of scale and pricing of utility cloud computing.

In addition to allowing the software to run on AWS, Teradata also announced optional advisory services to assist customers with the setup and configuration in AWS. The white-glove configuration allows customers a well-supported environment configured by professional with years of experience, rather than a do-it-yourself approach to getting your own analytics running in the cloud.

teredata1

Teradata Database on AWS intends allow companies of all sizes to run a powerful analytics platform, locating it closer to the datasets it needs to investigate while enabling adoption with a more approachable cost.

January 13, 2016 0 comments

Cloud Trends

Teradata brings new analytics capabilities to Amazon Web Services

by Philip Sellers October 20, 2015

written by Philip Sellers

Teradata has announced that it is bringing its Teradata Database data warehousing and analytics platform to the public cloud on Amazon Web Services. The move is a direct reaction to customer requests to bring the analytics platform to the cloud of their choice and provides customers with the option to adopt Teradata’s analytics services in a self-service cloud environment with pay-as-you-go pricing.

With the AWS offering, Teradata is looking to make its software more widely accessible to new customers, lower the barriers to entry, enable closer proximity to data sources and provide customers with easier scalability. In particular, the scalability with AWS allows for easier handling of peak workloads with hourly billing, differentiating the public cloud offering from Teradata Cloud.

Two years ago, Teradata opened up its Teradata Database and Teradata Aster Discovery Platform and made them available under the name Teradata Cloud. Teradata Cloud is a flat-rate subscription service that is intended to offload installation and maintenance of a Teradata implementation on premises. It maintains the same mission-critical level of service as an on-site implementation without the capital investments. Teradata Cloud includes enterprise-class security managed by Teradata along with services from the company to help setup and configure the environment. The appeal of a flat-rate cloud service is unlimited consumption. But for many customers, unlimited was beyond their requirements.

Teradata on Amazon Web Services allows a customer to order Teradata Database directly from the Amazon control panel and scale it accordingly within AWS. Unlike the Teradata Cloud offering, Teradata on AWS is intended for the customer who wants to take more responsibility for their cloud offering, for instance, the customer is responsible for tasks like security of the environment. But it offers much more flexibility to the customer to size the environment and pay only for what they consume.

teredata1

The AWS capabilities joins Teradata’s existing cloud offering within the Teradata Cloud and will be made available during the first quarter of 2016.

October 20, 2015 2 comments

Trends

Happy SysAdmin Day to my fellow SysAdmins!

by Philip Sellers July 31, 2015

written by Philip Sellers

Happy SysAdmin Day to my fellow SysAdmins! A lot has changed in the last decade and in celebration, here’s a fun comic from Solarwinds celebrating our work and how it has changed and will change in the future. Have a great day!

The Adventures of Dave - Englishjpg_Page1

July 31, 2015 0 comments

Trends

Techazine.com welcomes a new writer, Jason Foxworth

by Philip Sellers December 23, 2014

written by Philip Sellers

Several years ago, the branding for this site changed to Techazine.com after the domain became available. The idea then was to open up the site to additional voices and I’m really glad to say that today a new voice is joining the site.

Jason Foxworth, who is also a systems administrator, will begin writing at Techazine.com focusing on Microsoft, Citrix and VMware topics. I have had the privilege of working along side of Jason for many years at Horry Telephone and, now, he’s decided to add his voice to the community and add some value from his experiences. One of the things that we both share is a desire to help you avoid our own problems, so expect help and how-to articles from him along with relevant subjects for the time. You may find Jason on Twitter as @TheTechFox.

Meet Jason

Jason Foxworth is a systems administrator for Horry Telephone Cooperative focused in application delivery and client virtualization. His responsibilities for the co-op are around Microsoft technologies including Windows Server, Active Directory, SCCM, Exchange, and Sharepoint, the Citrix portfolio of XenApp, XenDesktop, XenServer, and Netscaler and VMware products. Jason has certifications for Citrix XenApp 6.5, Citrix XenDesktop 7, and is a Microsoft Certified IT Professional. Prior to HTC, Jason worked as a customer technology consultant for a national retailer. Outside of technology, Jason enjoys fishing, hiking and the outdoors.

December 23, 2014 0 comments

Datacenter Storage Trends

Peer Persistence and Adaptive Optimization interoperation on 3PAR

by Philip Sellers July 7, 2014

written by Philip Sellers

This is part one of a two part series on enhancements to the HP 3PAR StoreServ platform announced at HP Discover in June.

One of the biggest benefits of being an HP invited blogger for HP Discover is the opportunity to sit and talk with architects and executives during coffee talks and importu meetings in the bloggers lounge. This gives me and other bloggers the opportunity to ask our specific questions from really technical or strategic HP staff. During the past two HP Discover events, I have had great opportunities to talk to HP 3PAR storage team members and get some great information about specific issues and some ideas on where things may be heading.

At HP Discover in Barcelona back in December, Calvin Zito (@HPStorageGuy) introduced me to Siamak Nazari, an HP Fellow and chief architect for 3PAR’s OS. I and few other bloggers had the opportunity to talk about how HP develops the 3PAR OS and the evolution of the ASIC inside the 3PAR.

In December, I asked Nazari specifically about things that break with Peer Persistence and the 3PAR advanced features. Earlier in the year, my employer had implemented a pair of 3PAR StoreServ 7400’s in our environment in a Peer Persistence configuration. I quickly noticed that the advanced feature sets didn’t fully support Peer Persistence and I was curious when we might expect that.

Peer Persistence is 3PAR’s flavor of Metro Storage Clustering for VMware. Peer Persistence allows synchronous replication between the two arrays and a seamless failover from one set of LUNs to another between arrays without interruption to the VMware workloads. The primary 3PAR presents its LUNs under a WWN as active storage paths and the secondary/replication array presents its LUNs in a secondary/passive state until the failover occurs. When failover occurs, the replica becomes the active paths and the other array becomes secondary/inactive. The failover and monitoring can be automated with a Quorum Witness to handle the failover in the event of an array failure.

Peer Persistence is the new addition to the 3PAR feature set and understandably, a lot of the other advanced features don’t recognize Peer Persistence. One of the things we realized immediately is that Peer Persistence and Adaptive Optimization do not share data. So while AO is balancing data between storage tiers on the primary array based on the read and write characteristics of data, the secondary array is not receiving all the read operations so its AO will not optimize the same or as well as the primary.

Nazari acknowledged that this is the case in 3PAR at the present time. One of the potential solutions to this is an enhanced federation between 3PAR arrays, which may get delivered in a future 3PAR OS release. There is nothing technically to prevent this type of data sharing between arrays to allow AO to optimize workloads the same across arrays, but presently the arrays don’t share the data. Federation would allow for this type of data and a lot of other data to be shared and managed centrally across multiple arrays.

But one of the things that I’ve realized is that HP, like all companies, has limited resources to apply to a large set of problems to solve. So, they prioritize features based on biggest benefit to customers and other factors.

So during HP Discover in June, I had the opportunity to talk with Priyadarshi Prasad (@Priyadarshi_Pd) about developments announced during the June event, including thin deduplication and HP’s strategy to get solid state storage down to the cost of spinning disk and I also wanted to get an update on my earlier question.

I asked again about any developments for advanced features like Adaptive Optimization interoperating with Peer Persistence. This time, we had a different conversation – one that prompted this post.

Pointing back to the announcements of the latest HP Discover, Prasad talked with me about the total cost solid state disk in a 3PAR array. This June, HP announced the HP 3PAR StoreServ 7450 all flash array with new 1.9TB Solid State Disks, Thin Deduplication and Adaptive Sparing. As a combination of these announcements, HP estimates it can deliver an all-flash array with usable storage at a cost of $2 per GB – similar to the cost of spinning disk. Prasad pointed out that if HP can reduce the cost of solid state storage to the same as spinning disk, would technologies like Adaptive Optimization even be needed anymore? Spinning disk is adopted for its low cost per GB, but if solid state can be adopted at the same cost, why wouldn’t customers go there instead.

While delivering $2 per GB solid state, until there is another tier of storage that is more costly and better performing than solid state, tiering technologies would become more or less obsolete. One valid observation is that non-volitile RAM (NVRAM) is coming and may present options for another, fast tier of storage, so we may see this come again, but it sounds like today that federated AO may have just moved way down the list of priorities.

As a customer with existing 3PAR with spinning disk, obviously I want to see some sort of federation and central management which also shares AO characteristics between arrays. It doesn’t do anything for customers who have already adopted spinning disk on 3PAR. Most likely, if we adopted all solid state, it will be in a new array or within a tier on our existing arrays. Its not likely for us or many customers to take existing spinning disk offline within existing arrays. So, for this reality to come to fruition, its going to take several years for arrays to refresh all-together or for new arrays to be adopted side by side.

But it certainly warrants more information and investigation to the HP claim of $2 per GB for solid state disk. That’s where part two in this series goes next – investigating HP’s 3PAR StoreServ $2 per GB.

Disclaimer: HP invited me to HP Discover as a blogger and covered my expenses to attend the event. I also spoke at a session during the event. As always, the views and opinions stated on this blog are my own. HP was given no editorial control over content or topics for my posts.

July 7, 2014 3 comments

Trends

Struggling with margin and technical debt

by Philip Sellers June 4, 2014

written by Philip Sellers

If left unchecked, its only a matter of time before every administrator becomes overwhelmed by technical debt. It may be just months on a new job for a lazy admin or it may be several years for someone whose actively trying to manage their task list, but at a certain point, its bound to happen. My coworkers are probably tired of me talking about the topic, but its one that’s really impacting me today. The idea of technical debt is the accumulation of all the things on your to-do list or your must-do list that cannot be done due to lack of time. This is an issue that affects so many environments – particularly in 2014 when all departments have been asked for the last 5 or 6 years to do more with less people and time.

Like other environments, we have adopted a number of new systems throughout the company. With each system and implementation, it comes with a whole list of things to startup the new system and then adds maintenance and upkeep tasks to our lists. As new projects were onboarded, time began to slip away for handling other maintenance tasks. And because of the fast pace keeping up with new projects, some tasks get left undone.

Any of these things that are left undone get added to your technical debt. Think of it the same way you think about credit card debt. Each time you have a task or a new project that needs something done its like swiping your credit card. When you complete the task, you’ve paid it off. Over time, you may forget a task or it simply drops off your to-do list but its still part of your owed balance.

There are a lot of books in the library that deal with time management, but I want to talk about a slightly different take on the problem from a systems administrator’s viewpoint. You see, to an administrator, managing technical debt is essential in keeping an environment secure and operating well. If you allow your technical debt to get too large, you are introducing lots of risk into your environment by ignoring the tasks that should be performed simply because you’ve exhausted your time to do these things.

So just like credit card debt, you need a plan to manage your technical debt. If left unmanaged, you may get into a situation where you struggle to make minimum payments – essentially being only able to react to the most pressing issue of the moment.

I can remember a time not long ago when I felt like my environment was pretty well managed. I remember having time to do a daily walk-through of our primary data center looking for warning or error lights on systems and I remember having time for doing periodic maintenance tasks on the systems I managed. Today, however, it seems I’m struggling just to make my minimum payments on technical debt.

So what is the plan? I think the first part of the plan is establishing margin.

Just like the margins on a page which help to make it more appealing to read, margin in your work life (or personal life) helps to bring you in from the edges and provide you with cushion. Without margins in a book, your eyes get tired faster because its more difficult to read without the room to breathe around the words on the page.

Margin allows you to react to the things that will inevitably come up while not ignoring the important things that need your attention. Margin is intentionally built in extra time that can be used for reacting or more importantly for dreaming about solutions to your bigger problems in the environment. Margin allows a relaxed pace at times to make better decisions about strategies in your environment, as opposed to only being allowed to make rapid fire reactions.

The second part of the plan is the Dave Ramsey Debt Snowball idea applied to technology. Dave Ramsey advocates picking the smallest of your credit cards and paying it off first rather than focusing on paying off the one with a higher balance or interest rate. The result is a quick win and a feeling of accomplishment. I think the same should be applied to technical debt. Pick a project with an undone task list and knock out that list. That feeling of accomplishment will help drive you towards the next and the next and the next things on your list.

But while I’m writing this, I’m really talking to myself as much as anyone else. I’m struggling to establish margin at work. The lack of margin today results in a more stressful workday where I feel constantly reactionary instead of proactive. I’m feeling like I’m treading water and about to sink. So, I’m interested to hear anyone else’s experiences or advice with technical debt.

June 4, 2014 1 comment

Datacenter Storage Trends

Sustainability takes stage at HP Discovery

by Philip Sellers December 10, 2013

written by Philip Sellers

HP announced its Earth Insight project, this morning, in conjunction with Conservation International during a press conference at HP Discover in Barcelona. HP Earth Insights is an initiative based on HP’s Vertica software that has created the Wildlife Picture Index (WPI) Analytics System. The WPI Analytics System interprets mass amount of photos the Conservation International’s Tropical Ecology and Assessment Monitoring (TEAM) network has received from its field cameras and makes the data usable by decision makers with the intent of driving sustainable practices.

Conservation International (CI) is a non-governmental, non-profit organization based in Arlington, Virginia, that focuses on environmental challenges and biodiversity loss. Although based in the US, the majority of its field work is done in other countries with a strong focus on protecting tropical biodiversity (the variation of life – both plants and animals). The application of big data tools to the information collected through TEAM is aimed at helping scientists understand why areas are losing biodiversity and take actions to prevent further loss. It helps scientists be able to analyze data from over 6,000 square kilometers of tropical forests.

“Sustainability is fundamentally about people’s relationship with nature,” Sandy Andelman, PhD, Conservation International’s Chief Scientist and Senior Vice President, said during today’s keynote onstage with CEO Meg Whitman. “If you’re going to manage that relationship effectively, we have to measure. And tropical forests are the part of the world with the biggest measurement gap.”

CI is trying to fill that information gap.

For CI, their system was previously a manual process of collecting, interpreting and analyzing the data from their TEAM camera traps. Now, with Vertica, CI has near real-time analytics from its collection of photographs to identify and interpret biodiversity loss. It changes both the pace and availability of that data needed to report. And Andleman says she hopes that this partnership will yield the data that countries will need to report again in 2020 and in additional that CI will understand how to manage the relationship between people and nature because of the data.

This morning was one of those mornings when I wished my wife was with me for an announcement. My wife, a university sustainability coordinator, is constantly educating me on the topic of sustainability, so some points of the announcement certainly resonate.

First, the project gives scientists additional data that they have never been able to have in the past. Although scientists previously had satellite imagery to observe loss of tropical forests and the loss of biodiversity within the forests, there has never been a tool for interpreting what’s going on under the canopy.

Second, the scientists of TEAM did not anticipate the overwhelming number of images that would be generated from the camera traps. They have over 1.5 million images and adding about 200,000 images each year. The manual process could not keep up – but the Vertica solution likely offers quick results as well as the ability to change how data is interpreted, sliced, grouped and summarized.

Third, from a sustainability practice, its all about understanding the human interaction with nature and once it is understood, then actions can be taken to minimize or lessen the impacts on nature to preserve the world and its resources for future generations.

For more information about the technical approach on the project, the WPI website has a great summary. You can also explore the results of the Vertica analytics through this dashboard on the WPI website. And lastly, HP launched a new HP Earth Insights website to provides a great overview and summary information from the project.

December 10, 2013 2 comments

Trends

Introducing Techazine

by Philip Sellers October 27, 2013

written by Philip Sellers

Welcome to Techazine, the new brand for my IT blog. The content is all still the same, but the name has just changed. Over the next several months, you will notice small and incremental changes as I roll out the Techazine branding. Unfortunately, my previous blog name, Tech Talk, had become far too generic since so many companies and blogs use it for their podcasts and web presence. I hope to keep all of the changes transparent to the reader and focus on building a name for the blog that people will remember.

Thanks for visiting and please follow in Twitter, LinkedIn for the latest updates.

~Philip

October 27, 2013 0 comments

Compute Datacenter Storage Trends

HP shares Big Data directions

by Philip Sellers September 16, 2013

written by Philip Sellers

bigdatalarge A couple weeks ago, I had the opportunity to attend a Big Data briefing from HP Chief Technologist for Data Management, Greg Battas. Battas is part of the newly formed Converged Systems group in HP. He was a pioneer of Very Large Databases (VLDBs) and analytics in the telecom industry who has worked in business intelligence and analytics for a couple decades. He speaks internationally on topics of data integration and holds several patents in the areas of Relations Database, parallel query optimization and real time infrastructure architectures.

Coming from a mid market company, Big Data seems like a problem that doesn’t affect me. Its a concept I have a difficult time wrapping my head around, and for that reason, I’ve not written about it in the past. But it seems that Big Data is more than just a buzzword of the year and there is a lot of innovation occurring around Big Data in attempts to solve customer problems with these datasets.

So, what is Big Data? Wikipedia seems to sum it up best by saying “Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications.” That helps.

So within the realm of Big Data, you begin to think about supercomputers, hyper-scale arrays of servers and new innovations like in-memory databases. So where is this going and what’s happening in Big Data today?

Direction: Consolidation of Clusters

Big Data Architectures in the past were based around very simple servers with direct attached storage. It leads to a first Hadoop cluster, then a second, third and fourth. A lot of the thought in Big Data was movement away from proprietary storage and databases with parallel programming and distributed file systems like Google File System on standard hardware. Common wisdom dictates that big data depends on massive IO to read huge amounts of data from disk and to optimize this you move compute closer to the data to reduce overhead. There are certain operations that benefit from being pushed down to the same location as the data.

In reality, what has been learned from Big Data workloads is that only a portion of the processing can be close to the data. Keeping data and compute local to one another is actually difficult and does not always result in optimal performance because the data may not be in the appropriate form for processing and may need to be shuffled or reduced. In addition, its observed that the majority of CPU power is still needed for analytics and aggregation. And when most work is done in-memory, the storage really doesn’t apply. This becomes particularly important with NVRAM and other persistent RAM technologies in the future. What all this allows is for the consolidation and re-tasking of clusters to meet multiple needs.

Direction: Software Defined Storage

Big Data workloads, particularly the largest datasets in the world, are running on distributed file systems on industry standard server hardware. These are parallel file systems rather than traditional storage arrays or databases. Hadoop Distributed File System is becoming a very common interface across multiple platforms and deployments and vendors are adopting HDFS and integrating it under their technologies. Today, there is a mix of proprietary and open source technologies.

HP is observing a lot of vendors running their proprietary systems on top of the HDFS, or a very similar parallel filesystem. HP’s internal blueprint with HAVEn is much the same. HAVEn has the unstructured data of Autonomy, the structured analytics of Vertica and running it on top of the HDFS. In addition to storage, the HDFS allows for data to be passed from one tool to another.

Direction: NoSQL is being adopted by a lot of software partners

The first wave of Big Data was around Batch where Hadoop was used for analytics and ETL offloading. It was often coupled with a company’s SQL databases. There is a growing interest in NoSQL products from independent software vendors because of a large dependence on the traditional database vendors. ISVs are seeing a large portion of their total sales being directed back to a database vendor at the close of a sale, so many are beginning to move their commercial products onto NoSQL products that do not require the large costs. It isn’t without challenges, since there is no longer a SQL interface or transaction management to move onto a NoSQL product that no longer requires the large costs of traditional databases. But there are several with active projects porting their products onto a NoSQL product like Hbase.

Direction: Shift to optimized hardware

With a specialized workload like Big Data, there is opportunity for tuning and tailoring hardware to specifically handle the work better than industry standard hardware. Within HP, Moonshot is one example of hardware that is hyper-scale, simple node architecture suited for Big Data (I previously covered Moonshot here). On each cartidge, co-processors or GPU’s could be added to better handle big data workloads, but there is even possibilities within the system-on-a-chip on the cartridge.

Battas mentioned the idea of Dark Silicon, or un-used and un-powered transistors within today’s chips. Today, the industry has the capability of packing on more transistors into a piece of silicon than we have connections to power, leaving them dark. But the interesting idea is tailoring the dark silicon cores to handle a specific task well and then rotating the customized core in and out of use to increase efficiency. This is a particularly interesting topic, enough so that I have written a second post today about dark silicon.

September 16, 2013 0 comments

Compute Datacenter Trends

HP and chipmakers exploring use of specialized dark silicon for improved performance and efficiency

by Philip Sellers September 16, 2013

written by Philip Sellers

I love when a briefing or conference call sends me searching for a new term I don’t recognize. I had that happen during an HP briefing with Greg Battas about Big Data last month. It was the first time I had heard of the concept of Dark Silicon, where there are more transistors on a chip than can be powered leaving some of them dark. To me, it sounded a lot like dark fiber – fiber laid in the ground but unlit because there is no use for it at the time.

When I began digging into Dark Silicon, there were relatively few sites with any information about it. Most sites that referred to Dark Silicon were from 2010 and 2011 and were referring to the pace as which chip makers can pack transistors on a chip was quickly outpacing the ability to keep them powered. Also, as the ability to pack more transistors increases, the size of the process stays about the same, leading to more and more cores being present in the same space. In many cases, the 4 and 8-core chips common in systems today physically have more cores on them, but cannot be powered. But at the same time, the cost of omitting these transistors is negligible for the manufacturer.

From the conference call I was on, Battas relayed to us that some interesting things are happening particularly in system on a chip designs. These applications are particularly power sensitive in mobile devices leaving more dark silicon in these devices. But the emerging idea is that instead of simply leaving the additional cores dark, these could be engineered for specific tasks and powered only when needed to improve the efficiency of a task.

Specialized Cores

I was able to find a great resource on the topic – a presentation titled “Is Dark Silicon Useful” from Michael B. Taylor, an associate professor at the University of California, San Diego. The presentation has a good general explanation of dark silicon and it details the topic that Battas mentioned on the call in more detail. The slide deck is available online and I suggest that you take a look at page 40 and beyond in particular.

The slide on page 43 is extremely interesting showing a 91% savings of energy through using conservation cores – which were specialty dark silicon cores tuned for a specific task, rather than traditional RISC based processing. One of the two insights that Taylor shares about conservation cores is that “specialized logic can improve energy efficiency by 10-1000x.” That’s pretty compelling.

In terms of HP and Big Data

In April, HP launched Moonshot for the world. Moonshot is a hyperscale, datacenter solution that uses server cartridges to massive scale out software. The cartridge (think blade, but more simplistic) includes all the workings of a server while offloading the power, cooling and networking to the enclosure.

After a couple briefings that have mentioned Moonshot, a few things become clear. First, Moonshot cartridges can be introduced on short, iterative cycles allowing for faster innovation and integration of the latest technologies. Second, the cartridges can be customized to perform specific tasks. One of the early briefings mentioned creating a cartridge GPU’s integrated on the board to handle facial recognition at ATMs. There is unique flexibility in that design.

Now bringing in the idea of dark silicon on Moonshot cartridges, designers and engineers are thinking that creating silicon specifically tuned to certain processes that can turn on and turn off regions as needed will improve efficiency even further with these system on a chip designs in the datacenter. There are applications like voice service processors which can greatly benefit of custom silicon.

“Because adding transistors to an existing piece of silicon is really cost effective, it lowers the barrier for putting really interesting acceleration onto the chip for databases,” according to Battas. HP really sees a lot of this innovation occurring from system on a chip vendors who are looking at applications for database accelleration. This could have large impacts in scale out architectures like Hadoop clusters and other Big Data applications.

In addition to the chip innovations, big data software is changing also as open source alternatives to the traditional big data software. This speeds adoption of new hardware innovations into the software is accelerated and this allows for new concepts to be rapidly adopted into platforms. For instance, Intel is working closely with the Apache Hadoop project to make sure that the software can take advantage of their hardware. Battas expects to see close collaboration by other big data software vendors and hardware vendors to try and exploit these types of increased performance with specialized hardware.

September 16, 2013 1 comment