Clouds and the governor

I’ve been meaning to respond to Monkchips speculation over IBM and Amazon from last year his follow-up why Amazon don’t need IBM. James and I met-up briefly before Christmas, the day I resigned from IBM UK but we ran out of time to discuss that. I wrote and posted a draft and never got around to finishing it, I was missing context. Then yesterday James published a blog entry entitled “15 Ways to Tell Its Not Cloud Computing”.

The straw that broke the camels back was today, on chinposin Friday, James was clearly hustling for a bite when he tweeted “amazed i didn’t get more play for cloud computing blog”.

Well here you go James. Your analysis and simple list of 15-reasons why it is not a cloud is entertaining, but it’s not analysis, it’s cheerleading.

I’m not going to trawl through the list and dissect it one by one, I’ll just go with the first entry and then revert to discussing the bigger issue. James says “If you peel back the label and its says “Grid” or “OGSA” underneath… its not a cloud.” – Why is that James? How do you advocate organizations build clouds?
Ahh yes, you don’t want them to do you? It’s all about utilizing Amazon, Google, Yahoo, Microsoft, SAP and a few other mega-organizations’ clouds. Let them find the coldest places on the planet, convince the locals they’ll be good for the local economy for that dozen or so IT Janitor positions, make a bundle off tax incentives and tax breaks and then custom building their own data centers, their way, using their plumbing, their own “value add”.

Yep, one day we might all run our enterprise apps in these data centers; in fact one day we might not have our own enterprise apps at all, we’ll all be running in some sort of egalitarian, googlistic “state” computing resource, where all your apps are provided, you Vulcan mind-transfer or teleport in, no API’s to use, no worries about troublesome transactions.

Information Technology is so easy when you look at it in one-dimension, or at best two. All you need are great developer tools and a cloud, clouds exist, and therefore they are. But is that practical for 99% of enterprise apps? One day it may be, but today it isn’t.

Quite a few years back, I was approached by a customer and asked to review their HLD for their very first grid. Yep, their idea was to CPU-scavenge other servers in their organization that were under-utilized to provide additional resources for an application that was costing them too much to run on their mainframe, and for which the demand was going through the roof.

When I reviewed the requirement, discussed it with them, looked at their plans to build the plumbing, what they planned to do with the app(convert from Smalltalk to Java), it just didn’t get the job done quickly enough, and the right return on investment.

What I proposed, and I can go back and check, but I figure it was mid-year 2003, was that they purchase an IBM BladeCenter; put it next to the mainframe in the data center; high-speed(then 1gige) interconnect; run Linux on the Blades with a provisioning engine; build plumbing to manage the scheduling and dispatching of the tasks(a 401k/pensions calculation) and move the app directly from the mainframe using SmallTalk, no rewrite.

Communication between the mainframe and the Grid, for yes, that’s what it really was, would be using a replacement stub on the mainframe transaction system, CICS aka the CICS Transaction Server, via MQ, into the grid dispatcher. This would select a blade, schedule the data to be loaded to the blade, then schedule and dispatch the SmallTalk VM.

If there was a failure for some reason and a blade didn’t respond in a timely, the scheduler would recover by dispatching another blade. Once a successful response had been received, it would be fed back to the mainframe using the same MQ infrastructure. Job done.

New capacity could be added as needed, plug a new blade into the blade centre; add a whole new blade center; no new software each time, just additional dispatching node; all operations managed and deployed as part of their normal process. The whole thing was built around a Grid, and my recommendation was to use OGSA. However, since I didn’t have time to implement it, and the customer was a little concerned about an open source solution they went with a proprietary grid solution over the same hardware using MQ and SmallTalk as recommended. You can go read about the success they had, that company was Hewitt, that 401k application was IBM’s.

My point in discussing that was to get context. Using James list, I would say that contravenes maybe 10 of James 15 reasons why it would be a cloud. Was it a cloud? No, it was a grid.

If a customer came to me today and asked me for a cloud, to do the same for an application which they had fiduciary and regulatory responsibility for the data and which was solidly transactional. The job has to be done once and precisely once. Oh, and this time the environment where the applications are deployed has to be generic, not specific to that app.

Today I’d approach the problem in much the same way. Maybe that marks me out as part of the problem, but I don’t think so. For all the really bright, really innovative solution builders there are out there, for the big thinkers for whom Monkchips and to a degree his Redmonk partners in crime, have become cheerleaders for, there have to be those of us who can take a more practical, parochial view. Managing a service in the sky might sound fun with drupal, being able to moan on your blog about a half day server outage, while having no other control or input might be cool, but it ain’t business.

For most businesses over the next 5-years, building clouds is a matter of being able to build a scalable, reusable compute infrastructure. One that is reusable, not tied to a specific vendor or platform that has it’s own management or plumbing infrastructure. If I had to build that today, I’d do it around a WSRF/WDSM base. Yes the logical extensions of OGSA. The point here though isn’t to force apps to code to OGSA/WS* etc. That’s how the infrastructure gets managed, scheduled, provisioned, and dispatched. It’s not how the apps communicate. And yes, it would be a cloud or a grid, or whatever you want to call it, most of al it would be a practical solution meeting the customers requirements.

The problem with the whole debate over WS* over the past few years is that people forgot why it where it started. Everyone got engaged in a jihad over complexity or simplicity. Me, I say go ahead, build your apps using the appropriate languages and tools, using the preferred databases, access and inter-process communications.

Just give me a dedicated subnet, an open pub/sub infrastructure that I can use to manage infrastructure aka the plumbing. If the customer asked me to integrate their “cloud” with, I’d add a bus. My bus would do message transforms, it would have on-ramps and off-ramps, we used to call them user exits, to allow easy customization and transformation. And no, this isn’t SOA. It’s just an open approach to build infrastructure.

So, that’s my context. It is a grid underneath, but once in place, it’s transparent to the apps. Most customers have to submit a requirements doc for it, because their purchasing rules force the vendor to reply in very formal terms. You bought hardware because you are using this as the prototype to re-architect your systems, not because you have to, but because you want to. You know where the systems are, and that’s the way you wanted it. There is a consultant in the room, because you saved a bundle by moving most of your day to day operational staff to more cost effective locations, often along with the systems themselves.

I could buy it on a credit card. IBM would likely default on the expense claim and refuse it, but then I had to buy my own Thinkpad docking station this year for the office!(good news, expenses down, profit up!). It doesn’t take more than 10-minutes to provision or deprovision once in place and is often much quicker. If it does take longer, you know who to call, how to fix it, and how much it will cost. I can decide if I want to connect to it from my computer, I’m not forced to, to use it.

The one thing I don’t get James, is why should it run more than one operating system? Who cares how many or what operating systems it runs? Is there some reason all the nodes shouldn’t run Linux? On the other hand, why force customers to re-architect their systems and run more than one operating system? If you have an open, platform independent infrastructure technology and a bunch of machines of the same or different types, the machines get scheduled, provisioned and dispatched based on the service they can offer, not based on what operating system they run. Colour me baffled by that one.

Being a pied piper is a fine ambition, in this context though, I’m not sure who the rats are!
p.s. James, if you get to read this over the weekend, I’ll be driving up to London and free Monday evening before heading down to Gatwick. I’m seeing my eldest for Breakfast Tuesday morning and then flying back to Austin. I’d have got in touch sooner but I’ve been busy seeing my kids too!