Last week, I attended HP’s Converged Cloud Tech Day in Puerto Rico. Fellow colleagues attended from North, Latin and South America. The purpose of the event was to 1) take a deep dive into HP’s cloud offerings and 2) visit HP’s Aguadilla location, which houses manufacturing and an HP Labs presence. What makes the story interesting is that HP is a hardware manufacturer, a software provider and a provider of cloud services. Overall, I was very impressed by what HP is doing…but read on for the reasons why…and the surprises.
HP Puerto Rico
HP, like many other technology companies, has a significant presence in Puerto Rico. Martin Castillo, HP’s Caribbean Region Country Manager provided an overview for the group that left many in awe. HP exports a whopping $11.5b from Puerto Rico or roughly 10% of HP’s global revenue. In the Caribbean, HP holds more than 70% of the server market. Surprisingly, much of the influence to use HP cloud services in Puerto Rico comes from APAC and EMEA, not North America. To that end, 90% of HP’s Caribbean customers are already starting the first stage of moving to private clouds. Like others, HP is seeing customers move from traditional data centers to private clouds to managed clouds to public clouds.
Moving to the Cloud
Not surprisingly, HP is going through a transition by presenting the company from a solutions perspective rather than a product perspective. Shane Pearson, HP’s VP of Portfolio & Product Management explained that “At the end of the day, it’s all about applications and workloads. Everyone sees the importance of cloud, but everyone is trying to figure out how to leverage it.” By 2015 the projected markets are: Traditional $1.4b, Private Cloud $47b, Managed Cloud $55b, Public Cloud $30b for a cloud total of $132b. In addition, HP confirmed Hybrid Cloud approach as the approach of choice.
While customers are still focused on cost savings as the primary motivation to move to cloud, the tide is shifting to business process improvement. Put another way, cloud is allowing users to do things they could not do before. I was pleased to hear HP offer that it’s hard to take advantage of cloud if you don’t leverage automation. Automation and Orchestration are essential to cloud deployments.
HP CloudSystem Matrix
HP’s Nigel Cook was up next to talk about HP’s CloudSystem Matrix. Essentially, HP is (and has been) providing cloud services across the gamut of potential needs. Internally, HP is using OpenStack as the foundation for their cloud service offering. But CloudSystem Matrix provides a cohesive solution to manage across both internal and external cloud services. To the earlier point about automation, HP is focusing on automation and self-service as part of their cloud offering. Having a solution that helps customers manage the complexity that Hybrid Clouds presents could prove interesting. Admittedly, I have not kicked the tires of CloudSystem Matrix yet, but on the surface, it is very impressive.
During the visit to Aguadilla, we joined a Halo session with HP’s Christian Verstraete to discuss architecture. Christian and team have built an impressive cloud functional reference architecture. As impressive as it is, one challenge is how to best leverage such a comprehensive model for the everyday IT organization. It’s quite a bit to chew off. Very large enterprises can consume the level of detail contained within the model. Others will need a way to consume it in chunks. Christian goes into much greater depth in a series of blog entries on HP’s Cloud Source Blog.
HP Labs: Data Center in a Box
One treat on the trip was the visit to HP Labs. If you ever get the opportunity to visit HP Labs, it’s well worth the time to see what innovative solutions the folks are cooking up. HP demonstrated the results from their Thermal Zone Mapping (TZM) tool (US Patent 8,249,841) along with CFD modeling tools and monitoring to determine details around airflow/ cooling efficiency. While I’ve seen many different modeling tools, HP’s TZM was pretty impressive.
In addition to the TZM, HP shared a new prototype that I called Data Center in a Box. The solution is an encapsulated rack system that supports 1-8 racks that are fully enclosed. The only requirement is power and chilled water. The PUE numbers were impressive, but didn’t take into account every metric (ie: the cost of chilled water). Regardless, I thought the solution was pretty interesting. The HP folks kept mentioning that they planned to target the solution to Small-Medium Business (SMB) clients. While that may have been interesting to the SMB market a few years ago, today the SMB market is moving more to services (ie: Cloud Services). That doesn’t mean the solution is DOA. I do think it could be marketed as a modular approach to data center build-outs that provides a smaller increment to container solutions. Today, the solution is still just a prototype and not commercially available. It will be interesting to see where HP ultimately takes this.
I was quite impressed by HP’s perspective on how customers can…and should leverage cloud. I felt they have a healthy perspective on the market, customer engagement and opportunity. However, I was left with one question: Why are HP’s cloud solutions not more visible? Arguably, I am smack in the middle of the ‘cloud stream’ of information. Sure, I am aware that HP has a cloud offering. However, when folks talk about different cloud solutions, HP is noticeably absent. From what I learned last week, this needs to change.
HP’s CloudSystem Matrix is definitely worth a look regardless of the state of your cloud strategy. And for data center providers and service providers, keep an eye out for their Data Center in a Box…or whatever they ultimately call it.
Between natural disasters like Hurricanes Sandy and Irene or man-made disasters like the recent data center outages, disasters happen. The question isn’t whether they will happen. The question is: What can be done to avoid the next one? Cloud computing provides a significant advantage to avoid disaster. However, simply leveraging cloud-based services is not enough. First, a tiered approach in leveraging cloud-based services is needed. Second, a new architectural paradigm is needed. Third, organizations need to consider the holistic range of issues they will contend with.
Technology Clouds Help Natural Clouds
If used correctly, cloud computing can significantly limit or completely avoid outages. Cloud offers a physical abstraction layer and allows applications to be located outside of disaster zones where services, staff and recovery efforts do not conflict.
- Leverage commercial data centers and Infrastructure as a Service (IaaS). Commercial data centers are designed to be more robust and resilient. Prior to a disaster, IaaS provides the ability to move applications to alternative facilities out of harms way.
- Leverage core application and platform services. This may come in the form of PaaS or SaaS. These service providers often architect solutions that are able to withstand single data center outages. That is not true in every case, but by leveraging this in addition to other changes, the risks are mitigated.
In all cases, it is important to ‘trust but verify’ when evaluating providers. Neither tier provides a silver bullet. The key is: Take a multi-faceted approach that architects services with the assumption for failure.
Changes in Application Resiliency
Historically, application resiliency relied heavily on redundant infrastructure. Judging from the responses to Amazon’s recent outages, users still make this assumption. The paradigm needs to change. Applications need to take more responsibility for resiliency. By doing so, applications ensure service availability in times of infrastructure failure.
In a recent blog post, I discussed the relationship cloud computing provides to greenfield and legacy applications. Legacy applications present a challenge to move into cloud-based services. They can (and eventually should) be moved into cloud. However, it will require a bit of work to take advantage of what cloud offers.
Greenfield applications, on the other hand, present a unique opportunity to fully take advantage of cloud-based services…if used correctly. With Hurricane Sandy, we saw greenfield applications still using the old paradigm of relying heavily on redundant infrastructure. And the consequence was significant application outages due to infrastructure failures. Consequently, greenfield applications that rely on the new paradigm (ie: Netflix) experienced no downtime due to Sandy. Netflix not only avoided disaster, but saw a 20% increase in streaming viewers.
Moving Beyond Technology
Leveraging cloud-based services requires more than a technology change. Organizational impact, process changes and governance are just a few of the things to consider. Organizations need to consider the changes to access, skill sets and roles. Is staff in other regions able to assist if local staff is impacted by the disaster? Fundamental changes from change management to application design processes will change too. And at what point are services preemptively moved to avoid disaster? Lastly, how do governance models change if the core players are out of pocket due to disaster? Without considering these changes, the risks increase exponentially.
So, where you do you get started? First, determine where you are today. All good maps start with a “You Are Here” label. Consider how to best leverage cloud services and build a plan. Take into account your disaster recovery and business continuity planning. Then put the plan in motion. Test your disaster scenarios to improve your ability to withstand outages. Hopefully by the time the next disaster hits (and it will), you will be in a better place to weather the storm.
Last month, I wrote The Future Data Center Is… and alluded to a shift in demand for data centers. Just to be clear, I don’t believe data center demand is decreasing. Quite the contrary, I believe demand is exploding! But how is demand for data centers going to change? What does the mapping of organizations to services look like?
First, why should you care? Today, the average PUE of a data center is 1.8. …and that’s just the average. That’s atrocious! Very Large Enterprises are able to drive that to near 1.1-1.3. The excess is a waste of energy resources. At a time when Corporate Social Responsibility and carbon footprint are becoming more in vogue in the corporate arena, data centers are becoming a large target. So efficiency matters!
Yesterday, I presented a slide depicting the breakdown of types of organizations and (respectively) the shift in demand.
It is important to understand the details behind this. To start, let’s take a look at the boundary situations.
SMB/ Mid-Tier Organziations
Data center demand from SMB and Mid-Tier organizations starts to shift to service providers. Typically, their needs are straightforward and small in scale. In most cases, they use a basic data center (sometimes just a closet) supporting a mixed workload running on common off-the-shelf hardware. Unfortunately, the data centers in use by these organizations are highly inefficient due to their small scale and lack of sophistication. That’s not the fault of the organization. It just further supports the point that others can manage data centers more effectively than they can. Their best solution would be to move to a colocation agreement or IaaS provider and leverage SaaS where possible. That takes the burden off those organizations and allows them to focus on higher value functions.
Very Large Enterprises (VLE)
At the other end of the spectrum, Very Large Enterprises will continue to build custom solutions for their web-scale, highly tuned, very specific applications. This is different from their internal IT demand. See my post A Workload is Not a Workload, is Not a Workload where I outline this in more detail. Due to the scale of their custom applications, they’re able to carry the data center requirements of their internal IT demand at a similar level due to their scale. If they only supported their internal IT demand, their scale would pale in comparison and arguably, so would their efficiency.
In some ways, the VLE without the web-scale custom application is a typical Enterprise with a mixed workload. Enterprises sit in the middle. Depending on the scale of the workloads, characterization, organization and sophistication, enterprises may leverage internal data centers or external ones. It’s very likely they will leverage a combination of both for a number of reasons (compliance, geography, technical, etc). The key is to take an objective view of the demand and alternatives.
The question is, can you manage a data center more effectively and efficiently than the alternatives? Also, is managing a data center strategic to your IT strategic initiatives and aligns with business objectives? If not, then it’s probably time to make the shift.
The Green Grid: Metrics and Measurements
Many folks want to look in a crystal ball and magically profess what the future looks like. In the land of technology, it’s not that easy. Or is it? Sure, we do have the ability to control our destiny. We are limited by our own boundaries…artificially set or not. This may seem fairly straight forward, but it’s not. Businesses are looking for technology organizations to evolve and change. Even if that means they shift how they use services and applications on their own. Hence shadow IT.
Over the course of my career, I’ve seen many data centers in various countries. Even today, the level of sophistication varies greatly with Switch’s primary Las Vegas data center at one end of the spectrum and a 20-year old data center from a top data center/ cloud provider at the other end. I’ll leave them unnamed to avoid any potential embarrassment. To contrast, I’ve toured newer data centers in their portfolio that are much more innovative.
The advent of cloud computing has flipped the way computing resources are used on it’s head. How data centers are used is changing quickly. And what’s inside is becoming more relevant to those that manage data centers, but less relevant to those who use them. Let me explain.
Operating a data center is complex. It is no longer just four walls with additional power and cooling requirements. To add complexity, the line between facilities and IT has blurred greatly. How does an organization deal with this growing complexity on top of what they’re already dealing with? Furthermore, as the complexity of the applications and services increases, so do the expertise requirements within the organization. How is every company that currently operates a data center expected to meet these growing requirements? In reality, they can’t.
Only those that are able to bring the scale of their applications and services will warrant the continued operation of their facility. General purpose IT services (core applications, custom applications and the like) will move to alternative solutions. Sure, cloud is one option. Co-location is another. There are many clever solutions to this growing challenge. Are there exception cases? Yes. However, it is important to take an unbiased view at the maturing marketplace and how to best leverage the limited resources available internally.
In summary, unless you are 1) operating applications or services at scale or 2) have a specific use-case, possibly due to regulatory or compliance requirements, or 3) do not, realistically, have a viable alternative… then you should consider moving away from operating your own data center. The future data center for many is an empty one.
Building data centers in specific areas is nothing new. Data centers are large consumers of power. That’s not news either. Typically, data centers are located near sources of low-cost (and hopefully renewable) energy. Energy is a large portion of the overall data center operational costs.
But power isn’t everything. Two other major considerations are connectivity to a variety of major backbone providers and people. Yes, people. How many skilled workers are willing to take the risk and relocate to a rural area? If the job doesn’t work out, where do they go? There is a premium to relocate people, which factors against the power savings.
Two ways to address the people issue are 1) locate the data center in close proximity to other data centers and 2) architect for a truly lights-out operation to limit staffing requirements. It seems that both are not only possible today, but also being encouraged.
Major companies such as VMware, Intuit, Microsoft, Yahoo, Dell along with commercial providers have build data centers in the Wenatchee/ Quincy area of Central Washington State. The combined data centers comprise more than two million square feet of data center space. That’s quite a large footprint for such a rural area. More recently, Facebook located and Apple is locating a data center in the Prineville, Oregon area.
If your company does not have the scale for large data centers, there are still options. Commercial data center providers are locating data centers in the Wenatchee/ Quincy area. There is also a growing trend in the creation of data center “parks”. These are locations that are specifically built to take advantage of power, cooling, tax implications and connectivity options. In addition, they’re close enough to metro areas to attract the talent required for operations.
I would expect to see an increase in data centers popping up in these data center parks and away from metropolitan areas where rent and power is expensive. In addition, cloud computing will only increase the movement of data center functions away from traditional approaches to commercial offerings in remote areas.
Could they? Could it be out of Vogue to operate your own data center? Current developments in Corporate Social Responsibility and a maturing data center marketplace are starting to drive these changes.
For many, this could be a discussion about the pink elephant in the room. Data centers have been, and continue to be a requirement for businesses around the world. We rely more heavily every day on systems and the applications they run. Those applications run on servers and use storage subsystems; all of which are connected with networking devices. Collectively, we call this the “IT Load” for a data center.
The root question is not whether a data center is required. The obvious answer is: Yes! The real question is: Do I need to operate my own data center? But we will get to answer that question in a minute.
Data Center Energy Consumption
Data centers are consuming a larger percentage of the world’s energy every day. Our growing appetite will continue to take a toll on natural resources. In 2007, the EPA issued (for some) an eye-opening report on data center consumption and potential areas of efficiency.
While the report is a bit dated (2007), the core data still holds true today. The majority of the report is focused on projections and potential areas of efficiency. In 2011, Jonathan Koomey issued an updated report on the findings.
In his report, he noted that data center power consumption did not grow as strongly as the EPA projected. By 2010, global data center energy consumption hit 1.3% while in the US that number rose to 2%. Those are still very significant numbers.
What is Missing?
To add more fuel to the figures, a significant number of “facilities” are missing. Most notably missing from these findings is the energy consumption by the myriad of smaller “data centers”. While many would not call them data centers, they still serve the same purpose of housing servers, storage and networking equipment. These are smaller closets, rooms and labs. It may be as small as a server and switch under a desk to a rack or two of gear in a closet to a 1,000 sqft room. It is much harder to pin down the power consumed by each of these smaller locations. If you consider that these are the common solution for Small and Medium Businesses (SMB), the aggregate consumption is significant.
Increasing the efficiency of the physical data center is a great start. There are many opportunities to improve the efficiency of power and cooling systems. People have focused on increasing the efficiency of power and cooling systems for years. Many of the solutions are simple to implement and make a significant impact. While others take quite a bit of work, expertise and money. And there are many brilliant minds around the world that are currently working on this very challenge.
However, the largest potential impact may come from the IT load itself. For the majority of IT loads, the equipment is not used efficiently. Server, storage and network utilization figures are much lower than they could be. Servers are designed (from an energy perspective) for high utilization. One look at the power supply power curve for a server supports this. On the server, processor utilization rates commonly peak at 20-30% with average utilization in the 5-10% range. In addition, the current implementation rates for virtualization are still relatively low. The latest figures suggest that as many as 50% of servers are virtualized. Anecdotally, that figure still seems high. Regardless, pushing the implementation of virtualization to 80%+ would significantly reduce the overall power consumption…for the same IT workload.
Imagine reducing the US power consumption by a full 1%. The impact could be that significant.
Strategic and World-Class Expertise
Now back to the root question: Do you need to have your own data center? Before answering, two other questions will shed light on the answer. Is your organization in a position to operate your data center (100,000 sqft facility, 5,000 sqft room, closet, lab, etc) at a world-class level? Asked a different way: Is your organization willing to make the investment of installing a team of people to operate a world-class facility where it is their whole job, not just a line in the job description? Second, is operation of a data center strategic to your organization? We already covered that data centers are vitally important. So is electricity. Are you willing to make the investment in operating a data center that is unique and provides an advantage from your competition? Or are there alternatives that better fit the strategic direction of the organization?
If you set personal beliefs, cultural norms and inertia aside, for most, the answer to these questions is no. There are viable alternatives today that offer the economics, flexibility and responsiveness. And the alternative data center providers do employ teams to ensure their facilities are world-class. Only those few with large-scale requirements or the uncommon corner case will still need to operate their own data center.
Cloud computing is just one of many ways to accomplish these objectives. Startups and others are already heading down this path unencumbered by cultural norms and inertia. The challenge for established organizations is how to effectively turn the corner.
Bottom Line: Most organizations are not in a position to efficiently operate a world-class data center and should look at alternative solutions. The data center provider market is mature and competitors are already heading down this path.
Today, Amazon suffered a major outage of their EC2 cloud services based out of their Virginia data center. There are plenty of other blogs with more technical details on what specifically took place. Many cloud pundits are pointing to the outage as another example of the immaturity of cloud-based infrastructures. I think this is overblown.
In past missives, I outlined examples of past outages:
- Oct 14, 2009 Microsoft Sidekick Data Loss
- Jun 29, 2009 Rackspace Data Center Outage
- May 14, 2009 Google Outage
- Mar 21, 2009 Carbonite Storage Failure
While the dust-up of Amazon is fresh, outages of infrastructure are something to expect. We expect them in our own data centers. So are we back to expecting a double standard with cloud providers? In the case of Amazon, is the expectation that a higher class of service is delivered for a fraction of the price compared with internally provided data center services? Really?
Outages happen all the time in cloud data centers. Most of those outages are never observed or significantly impact users. Why? In most cases, simple tiers of redundancy are used to lower the statistical probability that an outage will occur. Yes, I said lower the statistical probability…not eliminate it. That’s all that redundancy does.
Then why did these outages happen in such a large and public cloud offering? At some point, one has to make a business decision as to how much redundancy is valuable. It’s easy to take pot shots from the outside of a cloud provider and looking inward. But these same challenges exist within traditional data centers too. And not all redundancy is infrastructure-based. Application architectures must consider the risks too.
I submit that it is time that we need to consider a different approach to how we provide services. I’m not referring to IaaS services. I’m referring to application-level services (SaaS in many ways). Our application architectures have relied on redundant infrastructure at the most basic levels for some time. That includes networks, servers, storage and so on.
This may sound like a pipe dream, but application awareness needs to move much higher in the OSI stack. If you think about it, SaaS applications do this to some degree. Do you know which data center is serving data when visiting http://www.google.com/? No. But when you put that in your browser, it works. Why is that? Does that mean that Google doesn’t have infrastructure failures? Do they have applications failures? Of course they do. But they’ve architected their applications and infrastructure to be resilient from failures.
In the case of the Amazon failure today, if the client applications were architected to leverage multiple Amazon data centers, would they have experienced an outage? While it may not have eliminated the entire outage for clients, it most likely would have reduced the impact. From the initial reports, the outage appears to be isolated to Amazon’s Virginia data center.
Some will argue that data sets are the Achilles Heel and prevent this type of redundant application architecture. I would propose that maybe we just haven’t figured out how to deal with it yet.
Bottom line: Failures are a reality in private data centers and in the cloud. We need to stop fearing failure and start expecting it. How we prepare our services and applications to respond to failure is what needs to change.
In the spirit of paradigm shifts, here’s one to think about.
Servers that are purchased today will not be replaced. Servers have a useful lifespan. Typically that ranges in the 3-5 years depending on their use. There are a number of factors that contribute to this. The cost to operate the server grows over time and it becomes less expensive to purchase a new one. The performance of the server is not adequate for newer workloads over time. These (and others) contribute to the useful lifespan of a server.
At the current adoption rates of cloud-based services, said servers will not be replaced. But rather, the services provided from those systems will move to cloud-based services. Of course there are corner cases. But as the cloud market matures, it will drive further adoption of services. Within the same timeframe, when existing servers become obsolete, many of those services will move to cloud-based services.
This shift requires several actions depending on your perspective.
- Server/ Channel Provider: How will you shift revenue streams to alternative offerings? Are you only a product company and can you make the move to a services model? Are you able to expand your services to meet the demand and complexities?
- IT Organizations: It causes a shift in budgetary, operational and process changes. Not to mention potential architecture and integration challenges for applications and services.
These types of changes take time to plan and develop before implementation. 3-5 years is not that far away in the typical planning cycle for changes this significant. The suggestion would be to get started now if you haven’t started already. There are great opportunities available today as a way to start “kicking the tires”.
I’ve been part of a consortium lead by the US DOE and US EPA to develop recommendations in an effort to drive up the number of data centers that measure and report their efficiency. Today, the group published the first phase targeted at dedicated data centers. Here’s the text along with the document:
On January 13, 2010, eight leading organizations that set or use data center energy efficiency metrics, including the 7×24 Exchange, ASHRAE, The Green Grid, Silicon Valley Leadership Group, U.S. Department of Energy’s Save Energy Now and Federal Energy Management Programs, U.S. Environmental Protection Agency’s ENERGY STAR Program, U.S. Green Building Council, and Uptime Institute, met to discuss these metrics for data centers and formed a task force to further define these measurements. This task force, which has met regularly since the initial summit, has prepared the attached Recommendations for Measuring and Reporting Overall Data Center Efficiency. The goal of this document is to recommend a consistent and repeatable measurement strategy that allows data center operators to monitor and improve the energy efficiency of their facilities. A consistent measurement approach will facilitate communication of Power Usage Effectiveness (PUE), the recommended energy efficiency metric from the agreed upon guiding principles, among data center owners and operators. More information about the guiding principles from the January meeting can be found here.
The attached document addresses how to measure PUE in dedicated data centers. The task force will continue to meet to develop recommendations for how to measure PUE for data centers in larger mixed-use facilities.
We hope that these recommendations help you understand how to measure and publish overall data center infrastructure energy efficiency, and we welcome any questions by e-mail to ENERGYSTARdatacenters@icfi.com.
Recently, the US Securities and Exchange Commission (SEC) offered both a view and a reminder on release of data related to climate change impact. The interpretation is located at:
For businesses, this means a greater focus on carbon impact and greenhouse gas emissions. For data center owners, it goes beyond efficiency on the data center. Energy consumers will need to understand the “carbon-makeup” of the energy streams they’re consuming. For example, is the source a coal-fired, natural gas or hydroelectric power plant? Or is the energy a mixture of different sources? Utilities like PG&E provide information on their energy mix via their website:
The US Environmental Protection Agency (EPA) has a wealth of information on their website at:
As agencies and the business community gains a greater awareness of climate change impact, we (in IT) must gain a greater understanding of our impact as well.