Different ways to react to outages in the cloud: Google vs Microsoft vs Amazon

Launched less than 48 hours ago, the Google App Engine cron support has already seen a long outage spanning around 9 hours:

The scheduled tasks service saw some down time last night from 11:30pm and 2:30am PST (GMT-8), and again from 6am to 12:00pm PST.

I must confess that I don't have a (big) problem with the fact that the service was down. Microsoft Azure has been down for 20+ hours sometimes in March and I've heard of quite a few Amazon AWS outages in the past. But this is not the subject of this post.

What I am actually interested in is to see how these teams are reacting when facing problems, so let's take a look at a couple of examples.

Google

Here is the fragment from the latest report from Google about the issue I've mentioned at the beginning of the post:

This issue has been identified and fixed.

And this is not the first time they are very short on providing any sort of explanation (I have quite a few more examples, but I'll keep it short)

Investigation Complete - Issue Resolved
March 10, 2009 at 1:46 am 2009-03-10T01:45:51Z 2009-03-10T01:45:51Z
Online
We have determined that this spike did not affect the performance or uptime of applications. If you feel we have incorrectly diagnosed this issue please inform us by posting in our developer forum.

Investigation Complete - Issue Resolved
March 9, 2009 at 12:47 am 2009-03-09T00:46:51Z 2009-03-09T00:46:51Z
Online
We have determined that this spike did not affect the performance or uptime of applications. If you feel we have incorrectly diagnosed this issue please inform us by posting in our developer forum.

Microsoft

Let's just compare the above reports with Microsoft's report about their Azure outage. The team has posted on their blog an explanation covering 3 aspects:

• What Happened?
• What Was Affected?
• How Will We Prevent This in the Future?

Leaving aside the fact that as a way to prevent further problems the Azure team has decided to provide by default a second instance that is not counted against the service quota:

For continued availability during upgrades, we recommend that application owners deploy their application with multiple instances of each role. We'll make two the default in our project templates and samples. We will not count the second instance against quota limits, so CTP participants can feel comfortable running two instances of each application role.

the report makes me feel confident in the team capability to deal with problems.

Amazon

Unfortunately I don't have any specific references here, but I clearly remember Amazon been very explicit in their explanations and the plans to address similar issues in the future.

My question now is: who would you trust?

I do feel that failing to provide any details about the problems and their resolution is not the best policy to convince your users about your capability to operate the system and its stability.

What we've got here is a failure to communicate.

Cloud Computing Unification Storm Calming Down

After what seemed to be the start of an interesting debate (with quite a few dramatic declarations), it looks like IBM and Microsoft sat at the same table and came up with a polite common position:

Representatives of CCIF, CloudCamp, Cisco, IBM, Intel, Microsoft and the IEEE-ISTO met while attending the Cloud Computing Expo in New York. Other companies were invited but were unable to attend, generally due to the short notice.

The companies agreed on a shared goal to promote use and awareness of open and interoperable cloud computing. The group brainstormed several ideas including the possibility to build on the momentum created by CloudCamp.

Another topic was the ability to enable participants, from individuals and companies, both large and small, to be able to contribute to and use the results of broad community collaboration. Additionally, the possibility of a trade association or marketing association for cloud computing was discussed but no specific actions were agreed. The final topic was the need to have broader participation from the community in this discussion

But don't let this declaration trick you. The fight for leadership is far for being over! In fact it is just starting and what happened last week was just a rehearsal. Interesting times ahead!

In case you missed the events, here is a series of post summarizing the week:

• Microsoft to lead the cloud computing market
• Cloud computing Unification: Microsoft or Sun?
• Cloud Computing Unification Getting Ugly

Resources:

• IBM And Microsoft Bury The Hatchet On 'Open Cloud'... Sort Of

Cloud Computing Unification Getting Ugly

Just a couple of days after I was reporting about the cloud computing unification manifesto and the debates it generated on the cloud computing market, it looks like we know have two manifestos:

The first one was posted by Geva Perry on his blog: The Open Cloud Manifesto: A Call to action for the worldwide cloud community

This document is intended to initiate a conversation that will bring together the emerging cloud computing community (both cloud users and cloud providers) around a core set of principles. We believe that these core principles are rooted in the belief that cloud computing should be as open as all other IT technologies.

This one seems to have been lead by IBM and it is signed by quite a few companies (naming just a few: IBM, Sun, VMWare, Cisco, SAP, AMD and other 30), except at least 4 big names: Amazon, Microsoft, Google, Salesforce.com.

The second one is hosted on Wikipedia and was created as a response to the Microsoft remarks about having this initiative being really open and collaborative:

Principles

• User centric systems enrich the lives of individuals, education, communication, collaboration, business, entertainment and society as a whole; the end user is the primary stakeholder in cloud computing.
• Philanthropic initiatives can greatly increase the well-being of mankind; they should be enabled or enhanced by cloud computing where possible.
• Openness of standards, systems and software empowers and protects users; existing standards should be adopted where possible for the benefit of all stakeholders.
• Transparency fosters trust and accountability; decisions should be open to public collaboration and scrutiny and never be made 'behind closed doors'.
• Interoperability ensures effectiveness of cloud computing as a public resource; systems must be interoperable over a minimal set of community defined standards and vendor lock-in must be avoided.
• Representation of all stakeholders is essential; interoperability and standards efforts should not be dominated by vendor(s).
• Discrimination against any party for any reason is unacceptable; barriers to entry must be minimised.
• Evolution is an ongoing process in an immature market; standards may take some time to develop and coalesce but activities should be coordinated and collaborative.
• Balance of commercial and consumer interests is paramount; if in doubt consumer interests prevail.
• Security is fundamental, not optional.

I'd say that it is more than clear that aligning the cloud computing initiatives will take a while and that the fight for the market is just starting. Not to mention the fact that most probably Sun will not stay aside and will start pushing for its Virtual Data Center API initiative too Update: it looks like Sun is in fact part of one of the manifestos.

Resources:

• The Open Cloud Manifesto: A Call to action for the worldwide cloud community
• Cloud Computing Manifesto - Cloud Computing Community Wiki
• The Open Cloud Manifesto now signed and delivered
• IBM leading 'Open Cloud Manifesto' charge

Cloud Computing Unification: Microsoft or Sun?

Looks like this is generating more discussions than I have initially though. You can check the last two links in the resource section for even more detailed coverage.

Even if the Sun cloud computing solution is not yet public, they made available the specifications (Tim Bray is the lead of the spec) under a commons creative license. Moreover they made it clear that Sun would be happy to have their specification becoming the standard for the cloud computing.

On the other hand, Microsoft has built their cloud platform Azure around some open standards too: web addressability, SOAP, XML, and REST and are stating that

Our vision in taking this approach was to ensure that the programming model was extensible and that the individual services could be used in conjunction with applications and infrastructure that ran on both Microsoft and non-Microsoft stacks.

Last but not least there is a cloud manifesto initiative that is taking shape on the Cloud Computing Interoperability Forum:

Our goal is to draft a document that clearly states we (including dozens of supporting companies) believe that like the Internet, the cloud itself should be open. The manifesto does not speak to application code or licensing but instead to the fundamental principles that the Internet was founded upon - an open platform available to all. It is a call to action for the worldwide cloud community to get involved and embrace the principles of the open cloud.

Now if you think about it, who wouldn't want to become the leader in the cloud computing market and coordinate and shape its future? (Forrester, Gardner and almost everybody else is speaking about the bright future of cloud computing). Most probably we will see once again some friction in the industry for the specification leadership role and for the

And the signs of the debate are already here, as Microsoft is already blaming others for not inviting them to the manifesto group (there are tons of links covering the subject in the resource section).

Recently, we've heard about a “Cloud Manifesto,” purportedly describing principles and guidelines for interoperability in cloud computing. We love the concept. We strongly support an open, collaborative discussion with customers, analysts and other vendors regarding the direction and principles of cloud computing. When the center of gravity is standards and interoperability, we are even more enthusiastic because we believe these are the key to the long term success for the industry, as we are demonstrating through a variety of technologies such as Silverlight, Internet Explorer 8, and the Azure Services Platform. We have learned a lot from the tens-of-thousands of developers who are using our cloud platform and their feedback is driving our efforts. We are happy to participate in a dialogue with other providers and collaborate with them on how cloud computing could evolve to provide additional choices and greater value for customers.

We were admittedly disappointed by the lack of openness in the development of the Cloud Manifesto.

I don't believe that either Sun's or Microsoft's current approaches can become the specification. There is indeed a lot to be learnt from their experience, but a specification will have to bring to the table all the companies involved in providing cloud computing solution plus the community. A specification created by providers only has no value with the consumers are not involved.

Resources:

• The Sun Cloud API
• Sun Cloud API: Is Simplicity Enough?
• Tim Bray Designs Sun's Virtual Data Center RESTful API
• Introducing the Open Cloud Manifesto
• Cloud Manifesto
• Moving Toward an Open Process on Cloud Computing Interoperability
• Microsoft Readies For Another Big Tech Battle
• Microsoft expresses outrage at secret 'Cloud Manifesto'
• Microsoft loudly disses secret 'Cloud Manifesto'
• Microsoft criticizes drafting of secret 'Cloud Manifesto'
• Microsoft Bashes Secret 'Cloud Manifesto'
• Amazon Web Services: No Open Cloud Manifesto for us (TechMeme)
• Introducing the Open Cloud Manifesto (TechMeme)

Tim Bray Designs Sun's Virtual Data Center RESTful API

Tim Bray speaking about the Virtual Data Center API specification goals:

This is a unified view of the storage and compute and networking services. It’s built around the notion of a “Virtual Data Center” (VDC), which contains networks and clusters of servers and public IP addresses and storage services. The idea is to give you the administrative and operational handles to build something really big and ambitious. The VDC notion is really slick and I think something like it is going to be required in any serious cloud API going forward.

Tim Bray is giving also a high level rationale for the RESTful API:

Why, then? Simply because we wanted a bits-on-the-wire interface. APIs, in the general case, suck; and are really hard to make portable. Bits-on-the-wire are ultimately flexible and interoperable. If you’re going to do bits-on-the-wire, Why not use HTTP? And if you’re going to use HTTP, use it right. That’s all.

Update InfoQ has published a couple of days ago an interview with Tim Bray on the Future of the Web (nb IMO a must see).

Resources:

• The Sun Cloud
• The Sun Cloud API
• Tim Bray on the Future of the Web

Microsoft to Lead the Cloud Computing Market

Not so long ago, some where saying that Microsoft is loosing the cloud computing bandwagon and that they have no idea what to do about it. Well, I'm pretty sure that now those that said it are biting their tongues. Not only that Microsoft has jumped into the cloud computing bandwagon with their Azure platform, but if they continue this way, I'd say that quite soon they will probably lead the market.

How sure am I about it? Nobody can be sure about anything these days, but I can tell you this: after spending time with both Amazon Web Services and Google App Engine, now I am preparing a virtual environment to start looking into Windows Azure.

Leaving aside my personal opinion, I'd encourage you to take a look at what Azure is offering (and what the plans are). Then do keep in mind that Microsoft has always been into integration and interoperability (in the sense that their products were always created to integrate and interoperate with existing ones). Last, but not least add to the mix the Microsoft BizSpark initiative (a program that between other things is offering startups 3 years free licenses to all Microsoft products) and you'll probably have a new perspective of the impact Microsoft will have on the cloud computing market.

Anyways, here are a couple of the latest announcements and presentations from Mix09 just to wet your taste. (nb: Silverlight required. Some of the videos are also available for download.)

• Overview of Windows Azure
• See through the Clouds: Introduction to the Azure Services Platform
• Building Web Applications with Windows Azure
• Windows Azure Storage
• What's New in Microsoft SQL Data Services
• Cloud Computing: What's in It for Me?
• Using the Windows Azure Tools to Build Cloud Services
• Simplifying Distributed Access Control with Microsoft .NET Services
• Mesh-Enabled Web Applications
• Live Framework and Mesh Services: Live Services for Developers
• Connecting Applications across Networks with Microsoft .NET Services
• A Lap around Microsoft .NET Services

Overview of Windows Azure

See through the Clouds: Introduction to the Azure Services Platform

Building Web Applications with Windows Azure

Windows Azure Storage

What's New in Microsoft SQL Data Services

Cloud Computing: What's in It for Me?

Using the Windows Azure Tools to Build Cloud Services

Simplifying Distributed Access Control with Microsoft .NET Services

Mesh-Enabled Web Applications

Live Framework and Mesh Services: Live Services for Developers

Connecting Applications across Networks with Microsoft .NET Services

A Lap around Microsoft .NET Services

I know this is a lot, but I'd strongly suggest to watch some of them in order to better understand why I think that Microsoft is going to be a leader in the cloud computing market.

Resources:

• RESTful Services for the Programmable Web with Windows Communication Foundation
• Caching REST with Windows Communication Foundation
• Developing RESTful Services and Clients with "M"
• Modeling RESTful Data Services: Present and Future

Cloud computing Links 2008-01-01

Links from DailyCloud: the daily coverage of the cloud computing market:

• Google's Will Power and Data Center Efficiency
• Go Higher with Cloud Computing
• Cloud Computing & Telephony: Twilio Shows EC2 and S3 in Action