Archive
How to Leverage the Cloud for Disasters like Hurricane Sandy
Between natural disasters like Hurricanes Sandy and Irene or man-made disasters like the recent data center outages, disasters happen. The question isn’t whether they will happen. The question is: What can be done to avoid the next one? Cloud computing provides a significant advantage to avoid disaster. However, simply leveraging cloud-based services is not enough. First, a tiered approach in leveraging cloud-based services is needed. Second, a new architectural paradigm is needed. Third, organizations need to consider the holistic range of issues they will contend with.
Technology Clouds Help Natural Clouds
If used correctly, cloud computing can significantly limit or completely avoid outages. Cloud offers a physical abstraction layer and allows applications to be located outside of disaster zones where services, staff and recovery efforts do not conflict.
- Leverage commercial data centers and Infrastructure as a Service (IaaS). Commercial data centers are designed to be more robust and resilient. Prior to a disaster, IaaS provides the ability to move applications to alternative facilities out of harms way.
- Leverage core application and platform services. This may come in the form of PaaS or SaaS. These service providers often architect solutions that are able to withstand single data center outages. That is not true in every case, but by leveraging this in addition to other changes, the risks are mitigated.
In all cases, it is important to ‘trust but verify’ when evaluating providers. Neither tier provides a silver bullet. The key is: Take a multi-faceted approach that architects services with the assumption for failure.
Changes in Application Resiliency
Historically, application resiliency relied heavily on redundant infrastructure. Judging from the responses to Amazon’s recent outages, users still make this assumption. The paradigm needs to change. Applications need to take more responsibility for resiliency. By doing so, applications ensure service availability in times of infrastructure failure.
In a recent blog post, I discussed the relationship cloud computing provides to greenfield and legacy applications. Legacy applications present a challenge to move into cloud-based services. They can (and eventually should) be moved into cloud. However, it will require a bit of work to take advantage of what cloud offers.
Greenfield applications, on the other hand, present a unique opportunity to fully take advantage of cloud-based services…if used correctly. With Hurricane Sandy, we saw greenfield applications still using the old paradigm of relying heavily on redundant infrastructure. And the consequence was significant application outages due to infrastructure failures. Consequently, greenfield applications that rely on the new paradigm (ie: Netflix) experienced no downtime due to Sandy. Netflix not only avoided disaster, but saw a 20% increase in streaming viewers.
Moving Beyond Technology
Leveraging cloud-based services requires more than a technology change. Organizational impact, process changes and governance are just a few of the things to consider. Organizations need to consider the changes to access, skill sets and roles. Is staff in other regions able to assist if local staff is impacted by the disaster? Fundamental changes from change management to application design processes will change too. And at what point are services preemptively moved to avoid disaster? Lastly, how do governance models change if the core players are out of pocket due to disaster? Without considering these changes, the risks increase exponentially.
Start Here
So, where you do you get started? First, determine where you are today. All good maps start with a “You Are Here” label. Consider how to best leverage cloud services and build a plan. Take into account your disaster recovery and business continuity planning. Then put the plan in motion. Test your disaster scenarios to improve your ability to withstand outages. Hopefully by the time the next disaster hits (and it will), you will be in a better place to weather the storm.
The Future Data Center Is… Part II
Last month, I wrote The Future Data Center Is… and alluded to a shift in demand for data centers. Just to be clear, I don’t believe data center demand is decreasing. Quite the contrary, I believe demand is exploding! But how is demand for data centers going to change? What does the mapping of organizations to services look like?
First, why should you care? Today, the average PUE of a data center is 1.8. …and that’s just the average. That’s atrocious! Very Large Enterprises are able to drive that to near 1.1-1.3. The excess is a waste of energy resources. At a time when Corporate Social Responsibility and carbon footprint are becoming more in vogue in the corporate arena, data centers are becoming a large target. So efficiency matters!
Yesterday, I presented a slide depicting the breakdown of types of organizations and (respectively) the shift in demand.
It is important to understand the details behind this. To start, let’s take a look at the boundary situations.
SMB/ Mid-Tier Organziations
Data center demand from SMB and Mid-Tier organizations starts to shift to service providers. Typically, their needs are straightforward and small in scale. In most cases, they use a basic data center (sometimes just a closet) supporting a mixed workload running on common off-the-shelf hardware. Unfortunately, the data centers in use by these organizations are highly inefficient due to their small scale and lack of sophistication. That’s not the fault of the organization. It just further supports the point that others can manage data centers more effectively than they can. Their best solution would be to move to a colocation agreement or IaaS provider and leverage SaaS where possible. That takes the burden off those organizations and allows them to focus on higher value functions.
Very Large Enterprises (VLE)
At the other end of the spectrum, Very Large Enterprises will continue to build custom solutions for their web-scale, highly tuned, very specific applications. This is different from their internal IT demand. See my post A Workload is Not a Workload, is Not a Workload where I outline this in more detail. Due to the scale of their custom applications, they’re able to carry the data center requirements of their internal IT demand at a similar level due to their scale. If they only supported their internal IT demand, their scale would pale in comparison and arguably, so would their efficiency.
Enterprises
In some ways, the VLE without the web-scale custom application is a typical Enterprise with a mixed workload. Enterprises sit in the middle. Depending on the scale of the workloads, characterization, organization and sophistication, enterprises may leverage internal data centers or external ones. It’s very likely they will leverage a combination of both for a number of reasons (compliance, geography, technical, etc). The key is to take an objective view of the demand and alternatives.
The question is, can you manage a data center more effectively and efficiently than the alternatives? Also, is managing a data center strategic to your IT strategic initiatives and aligns with business objectives? If not, then it’s probably time to make the shift.
Related Articles:
Mark Thiele: Measuring the Size of a Data Center – Yes, it Matters
The Green Grid: Metrics and Measurements
Sneak Peek of the Under The Radar Conference
The Under The Radar (UTR) Conference (http://www.undertheradarblog.com/) is tomorrow, April 26, 2012. UTR is the intersection of hot up-and-coming startups, investors and judging. If the reception tonight was any indication, the conference and presentations should be very interesting. Here’s a sneak peak of my take of the hot areas and companies to watch:
Application Development Solutions
A few companies are presenting their solutions in the mobile and security space. In the era of cloud computing, these are two hot buttons that enterprises and service providers alike need to be keenly aware of. The move of the information worker from a stationary device to a mobile device is in process. CoIT and BYOD are both serious factors to the movement. Likewise, using traditional security paradigms in the new model run into serious complications. Tools are needed to help organizations make this move while managing and securing environments.
Companies: BitzerMobile, Cabana, Duo Security, Fabric Engine, Framehawk, StackMob
Platforms and Infrastructure
Building applications on top of infrastructure is nothing new. In the cloud era, the architecture…and options open up quite a bit. The cloud market is starting to mature and value is moving from core infrastructure to platforms and on to applications. Leveraging hosted platforms does require a different paradigm to succeed. In addition, when considering apps at scale, automation and orchestration become even more important. This is a very broad area with quite a bit of specialization. Moving forward, integration in the space will be the key to success…along with some consolidation.
Companies: Appfog, AppHarbor, CloudBees, CloudScaling, Drawn to Scale, ionGrid, Iron.io, MemSQL, MongoLab, Nodejitsu, NuoDB, Piston Cloud, Puppet Labs, Sauce Labs, ScaleArc, Zadara Storage
Monitoring and Analytics
One of the most interesting areas is how data is used and analyzed. And then taking action based on the information gleaned from the data. Players in this space range from aggregating data to understanding and analyzing it. Value increases as the data is moved into analytics and ultimately business actions taken based on the intelligence. While there is quite a bit of specialization in this area at different levels (application monitoring/ performance management to analytics and intelligence), added value will come when these can be tied together to drive business decisions.
Companies: Chart.io, Cloudability, Cloudyn, Datadog, DataSift, Infochimps, Metamarkets, Nodeable, Sumo Logic, Tracelytics
Interesting Areas to Watch
In today’s marketplace, there are the future-state solutions and concepts. And then there are the real-world solutions that solve today’s problems. Both states need to be understood and the ball needs to be moved forward…and fast! The increased amplitude of mobile devices along with cloud computing bring applications at scale into the forefront. Orchestration and automation becoming hallmarks to success to up-level the conversation and value IT brings to organizations. Ultimately, the play will be with data and analytics. But today, there are more fundamental issues on the table.
Of course, that’s just a cursory review of the upcoming presentations from the UTR conference. Look for more details in the UTR Twitter stream (#UTRconf) and posts after the conference.
The Future Data Center Is…
Many folks want to look in a crystal ball and magically profess what the future looks like. In the land of technology, it’s not that easy. Or is it? Sure, we do have the ability to control our destiny. We are limited by our own boundaries…artificially set or not. This may seem fairly straight forward, but it’s not. Businesses are looking for technology organizations to evolve and change. Even if that means they shift how they use services and applications on their own. Hence shadow IT.
Over the course of my career, I’ve seen many data centers in various countries. Even today, the level of sophistication varies greatly with Switch’s primary Las Vegas data center at one end of the spectrum and a 20-year old data center from a top data center/ cloud provider at the other end. I’ll leave them unnamed to avoid any potential embarrassment. To contrast, I’ve toured newer data centers in their portfolio that are much more innovative.
The advent of cloud computing has flipped the way computing resources are used on it’s head. How data centers are used is changing quickly. And what’s inside is becoming more relevant to those that manage data centers, but less relevant to those who use them. Let me explain.
Operating a data center is complex. It is no longer just four walls with additional power and cooling requirements. To add complexity, the line between facilities and IT has blurred greatly. How does an organization deal with this growing complexity on top of what they’re already dealing with? Furthermore, as the complexity of the applications and services increases, so do the expertise requirements within the organization. How is every company that currently operates a data center expected to meet these growing requirements? In reality, they can’t.
Only those that are able to bring the scale of their applications and services will warrant the continued operation of their facility. General purpose IT services (core applications, custom applications and the like) will move to alternative solutions. Sure, cloud is one option. Co-location is another. There are many clever solutions to this growing challenge. Are there exception cases? Yes. However, it is important to take an unbiased view at the maturing marketplace and how to best leverage the limited resources available internally.
In summary, unless you are 1) operating applications or services at scale or 2) have a specific use-case, possibly due to regulatory or compliance requirements, or 3) do not, realistically, have a viable alternative… then you should consider moving away from operating your own data center. The future data center for many is an empty one.
The New Data Center Park Trend
Building data centers in specific areas is nothing new. Data centers are large consumers of power. That’s not news either. Typically, data centers are located near sources of low-cost (and hopefully renewable) energy. Energy is a large portion of the overall data center operational costs.
But power isn’t everything. Two other major considerations are connectivity to a variety of major backbone providers and people. Yes, people. How many skilled workers are willing to take the risk and relocate to a rural area? If the job doesn’t work out, where do they go? There is a premium to relocate people, which factors against the power savings.
Two ways to address the people issue are 1) locate the data center in close proximity to other data centers and 2) architect for a truly lights-out operation to limit staffing requirements. It seems that both are not only possible today, but also being encouraged.
Major companies such as VMware, Intuit, Microsoft, Yahoo, Dell along with commercial providers have build data centers in the Wenatchee/ Quincy area of Central Washington State. The combined data centers comprise more than two million square feet of data center space. That’s quite a large footprint for such a rural area. More recently, Facebook located and Apple is locating a data center in the Prineville, Oregon area.
If your company does not have the scale for large data centers, there are still options. Commercial data center providers are locating data centers in the Wenatchee/ Quincy area. There is also a growing trend in the creation of data center “parks”. These are locations that are specifically built to take advantage of power, cooling, tax implications and connectivity options. In addition, they’re close enough to metro areas to attract the talent required for operations.
Reno, Nevada
http://www.datacenterknowledge.com/archives/2010/11/15/large-reno-project-to-generate-its-own-power/
Colorado
http://www.datacenterknowledge.com/archives/2012/03/09/energy-park-proposed-at-nexus-of-fiber-power/
I would expect to see an increase in data centers popping up in these data center parks and away from metropolitan areas where rent and power is expensive. In addition, cloud computing will only increase the movement of data center functions away from traditional approaches to commercial offerings in remote areas.
What Can In-Flight Phone Service Teach Us About In-Flight Wi-Fi?
If you are like me, you spend quite a bit of time sitting on an airplane. Over the years, the amount of flight-time has increased too. At the same time our professional and personal lives are becoming “always on”. Meaning, that we have a growing need to stay connected…digitally. It could be to respond to email, check the latest updates from Facebook, follow a Twitter feed or surf the Internet. The direct solution appears to be in-flight Wi-Fi connectivity. Sure, in-flight phone service has existed for many years now. And, in the days of dialup Internet access, one could use the in-flight phone to connect to your Internet Service Provider (ISP) and download email. However, the in-flight Wi-Fi service, and phone service to some degree, has hit a bit of turbulence along the way. Let me explain.
History
Airfone (air-to-ground phone service) was originally developed in the 1970’s. In the 1980’s, planes offered the service to paying customers. It was simple to use. Just swipe your credit card in the built-in reader on the handset and make the call. The service, however, was very expensive. Of late, calls could cost as much as $5.00 per minute!
In the 1990’s you would see people using the phones every once in a while. But it was not frequent. In the past decade, service declined even further. In the past few years, the service has been sold to a LiveTV, an affiliate of JetBlue. While several carriers still have handsets installed, US Airways and Delta have removed the phones from their planes.
Wi-Fi Enabled Planes
Fast forward to 2012. Communications moved from voice to data. People rely on email as much as they did voice calls. In response, in-flight Wi-Fi services made their entrance onto everyday flights. Over time, more and more flights are including Wi-Fi service in the air. That is, the ability to connect your smartphone, tablet or laptop to the Internet via the plane’s Wi-Fi connectivity. Even though many planes have added this functionality, Wi-Fi equipped planes are not ubiquitous. Airlines have committed to adding Wi-Fi capabilities to more planes over the coming years. The two main providers are Gogo (http://www.gogoair.com/) and Row 44 (http://www.row44.com/). Row 44 is equipping Southwest Airline’s planes while Gogo is the provider for many of the other carriers providing in-flight Wi-Fi service.
In an age where most major carriers have filed for bankruptcy, additional services bring potential revenue streams and additional expenses. It is a risky gamble for airlines where every nickel, dime and dollar is scrutinized for operational efficiencies. But is simply installing the service enough?
Service Costs
The expenses to equip planes are one thing. Recovering those costs from paying customers is another. Over the 30 years that in-flight phone service was available, the costs continued to increase. At the same time, demand for in-flight phone service decreased. Costs going up and demand going down is a sure recipe for failure.
Currently, costs to use in-flight Wi-Fi can range from $5-$15 per flight depending on distance, carrier and service. That may not seem like a lot if your company is willing to reimburse the expense. It may also make sense if you consider the productive time while connected. But it depends on “what” you are using the connectivity for. If you are only surfing the web, checking Facebook or Tweeting about each cloud and city you pass, the cost may not be warranted.
In-flight phone service was more of a novelty or critical use-case that warranted the use. Current data shows in-flight Wi-Fi may be taking a similar path. While many may not be swayed to pay the $5-15 per flight, it is enough to keep most at bay. There are monthly service plans for frequent flyers. In the past year, providers did offer a promotion for free service over the holidays. But again, is it worth it? And will it be enough to offset the costs to the carrier/ provider? Or are we reliving in-flight phone service all over again without learning from the lesson?
Usage Statistics
It begs the question: Just how many people are actually taking advantage of the service? Recent reports show that use is still relatively low. It is probable that “how” the service is used coupled with service costs contribute directly to the actual usage. If the costs (to the consumer) were lower,
If those reports are correct, will service providers turn a profit and keep the service intact? Below, is an infographic from Gogo based on research they conducted during the first half of 2011. It is one perspective on how people are using in-flight Wi-Fi and with which devices.
Related Items
It would be amiss not to mention the related issues with in-flight Wi-Fi. One of the key issues is power. Unless you’re using a tablet or laptop with long battery life (and a fully charged battery), you are going to run out of juice. Having a power outlet available is not just a handy item, but necessary for most to fully take advantage of in-flight Wi-Fi for the entire flight. Unfortunately, not all planes include power ports. The ones who do offer power ports typically only offer them in Business Class or First Class. Of those that offer power in Economy Class, it may be every other row, only certain sections of Economy Class or every other seat. In addition, the type of connection may vary between proprietary EmPower outlets, 12v cigarette lighter style plugs or regular 110v US plugs.
Bottom Line: In-flight Wi-Fi has not hit the “sweet-spot” for price point vs. service use vs. availability. Until it reaches closer to that point, it will fail to gain significant altitude.
For further reading:
USA Today: Wi-Fi Service Slow to Take Off
Economist: Continued Unpopularity of In-Flight Wi-Fi
http://www.economist.com/blogs/gulliver/2011/12/flight-wi-fi
Verizon Cancelling In-Flight Phone Service
http://articles.latimes.com/2006/jun/27/business/fi-briefs27.3
JetBlue LiveTV to Buy Verizon’s Airfone Network
http://www.reuters.com/article/2008/06/09/us-jetblue-verizon-idUSN0944090420080609
Amazon Outage Concerns Are Overblown
Today, Amazon suffered a major outage of their EC2 cloud services based out of their Virginia data center. There are plenty of other blogs with more technical details on what specifically took place. Many cloud pundits are pointing to the outage as another example of the immaturity of cloud-based infrastructures. I think this is overblown.
In past missives, I outlined examples of past outages:
- Oct 14, 2009 Microsoft Sidekick Data Loss
- Jun 29, 2009 Rackspace Data Center Outage
- May 14, 2009 Google Outage
- Mar 21, 2009 Carbonite Storage Failure
While the dust-up of Amazon is fresh, outages of infrastructure are something to expect. We expect them in our own data centers. So are we back to expecting a double standard with cloud providers? In the case of Amazon, is the expectation that a higher class of service is delivered for a fraction of the price compared with internally provided data center services? Really?
Outages happen all the time in cloud data centers. Most of those outages are never observed or significantly impact users. Why? In most cases, simple tiers of redundancy are used to lower the statistical probability that an outage will occur. Yes, I said lower the statistical probability…not eliminate it. That’s all that redundancy does.
Then why did these outages happen in such a large and public cloud offering? At some point, one has to make a business decision as to how much redundancy is valuable. It’s easy to take pot shots from the outside of a cloud provider and looking inward. But these same challenges exist within traditional data centers too. And not all redundancy is infrastructure-based. Application architectures must consider the risks too.
I submit that it is time that we need to consider a different approach to how we provide services. I’m not referring to IaaS services. I’m referring to application-level services (SaaS in many ways). Our application architectures have relied on redundant infrastructure at the most basic levels for some time. That includes networks, servers, storage and so on.
This may sound like a pipe dream, but application awareness needs to move much higher in the OSI stack. If you think about it, SaaS applications do this to some degree. Do you know which data center is serving data when visiting http://www.google.com/? No. But when you put that in your browser, it works. Why is that? Does that mean that Google doesn’t have infrastructure failures? Do they have applications failures? Of course they do. But they’ve architected their applications and infrastructure to be resilient from failures.
In the case of the Amazon failure today, if the client applications were architected to leverage multiple Amazon data centers, would they have experienced an outage? While it may not have eliminated the entire outage for clients, it most likely would have reduced the impact. From the initial reports, the outage appears to be isolated to Amazon’s Virginia data center.
Some will argue that data sets are the Achilles Heel and prevent this type of redundant application architecture. I would propose that maybe we just haven’t figured out how to deal with it yet.
Bottom line: Failures are a reality in private data centers and in the cloud. We need to stop fearing failure and start expecting it. How we prepare our services and applications to respond to failure is what needs to change.
Servers Purchased Today Will Not Be Replaced
In the spirit of paradigm shifts, here’s one to think about.
Servers that are purchased today will not be replaced. Servers have a useful lifespan. Typically that ranges in the 3-5 years depending on their use. There are a number of factors that contribute to this. The cost to operate the server grows over time and it becomes less expensive to purchase a new one. The performance of the server is not adequate for newer workloads over time. These (and others) contribute to the useful lifespan of a server.
At the current adoption rates of cloud-based services, said servers will not be replaced. But rather, the services provided from those systems will move to cloud-based services. Of course there are corner cases. But as the cloud market matures, it will drive further adoption of services. Within the same timeframe, when existing servers become obsolete, many of those services will move to cloud-based services.
This shift requires several actions depending on your perspective.
- Server/ Channel Provider: How will you shift revenue streams to alternative offerings? Are you only a product company and can you make the move to a services model? Are you able to expand your services to meet the demand and complexities?
- IT Organizations: It causes a shift in budgetary, operational and process changes. Not to mention potential architecture and integration challenges for applications and services.
These types of changes take time to plan and develop before implementation. 3-5 years is not that far away in the typical planning cycle for changes this significant. The suggestion would be to get started now if you haven’t started already. There are great opportunities available today as a way to start “kicking the tires”.
Connectivity is the Cloud’s Achilles Heal…Sort Of.
Having written about the challenges for Road Warriors and cloud computing, I thought it a good opportunity to write about the challenges that connectivity brings to a cloud-based methodology. Some challenges are very real while others are perception.
It is true that road warriors are still challenged with hosting data in the cloud. There are a myriad of reasons this becomes problematic. The vast majority of airplanes still don’t have Internet connectivity. And those that do have limited bandwidth. Just try to open that large PowerPoint presentation or make a Skype call from an airplane over WiFi. The same is true in airports and hotels. And when there is connectivity, the options can be frustrating. I recently stayed at hotel in Japan (a global brand). They offered free WiFi in the lobby. In the rooms WiFi was not available…only wired access. Of course this presented a challenge for someone who typically only travels with an Apple iPad. And then there are the hotels, airports and other locations that charge exorbitant rates to use their WiFi network. If you’re traveling to a conference with other tech-minded folks, expect the connectivity performance to suffer accordingly.
However, for businesses, the challenges are less concerning. Businesses are more stationary and less mobile. Redundant connectivity is a real option as the cost of bandwidth drops. For this reason, connectivity is less of an issue for businesses than the mobile users that access cloud-based services. Local access to productivity apps and their related files can realistically be accessed across the Internet today. There are corner cases, but for the majority, it is very possible.
When considering whether to leverage cloud-based services, one has to consider many factors. Those include the location of users (end-users, developers, administrators), applications that consume the data and value to business (economic, flexibility and responsiveness). More to come on the “three tenants” of cloud computing in a future post.
It is important to consider the different scenarios. It is equally important to look at new paradigms and not be constrained by conventional wisdom.
U.S SEC Views on Disclosure Bring Focus to Carbon Impact
Recently, the US Securities and Exchange Commission (SEC) offered both a view and a reminder on release of data related to climate change impact. The interpretation is located at:
http://www.sec.gov/rules/interp/2010/33-9106.pdf
For businesses, this means a greater focus on carbon impact and greenhouse gas emissions. For data center owners, it goes beyond efficiency on the data center. Energy consumers will need to understand the “carbon-makeup” of the energy streams they’re consuming. For example, is the source a coal-fired, natural gas or hydroelectric power plant? Or is the energy a mixture of different sources? Utilities like PG&E provide information on their energy mix via their website:
http://www.pge.com/myhome/edusafety/systemworks/electric/energymix/index.shtml
The US Environmental Protection Agency (EPA) has a wealth of information on their website at:
http://www.epa.gov/climatechange/index.html
As agencies and the business community gains a greater awareness of climate change impact, we (in IT) must gain a greater understanding of our impact as well.
