Looking back, Unix System Services, Linux et al

I receive quiet a few emails weekly asking questions about product stratetgy and direction. Some of them are difficult to answer because they either delve into IBM confidential disclosures or are too complex to handle in email.

This one though was both interesting and bought back a flood of memories, I thought a post here was in order, with Dales permission.


Hey, thanks for getting in touch. I wouldn’t consider posting it to a blog without prior agreement, and actually on this topic, I don’t think I can be of much help.

I was VERY, active in the roll-out of USS in what was MVS/ESA 4.3, and the follow-on exploitation by the OS/390 web server and object oriented technology, that was some 12-years ago now. I was involved in the business case and strategy for Linux on the mainframe. You can still read my last contribution to the mainframe systems in the Linux for S/390 Redbook pointed to on my blog Books page.

However, shortly after that, I moved to IBM Software Group to help direct and communicate their overall Linux strategy and then in 2002 on to the design of the technologies behind IBM’s on demand initiative; Last year I moved to the AIX/System p UNIX team in Austin.

So, my following remarks are the way I see things now, from afar rather than as part of the zSeries team. Even when I was part of the team I was really only riding on the work done by others in determining the product need, doing the design for, and implementing the USS and ultimately LINUX on S/390.

First, [the UNIX Systems Services] USS was an important part of the revitalization of the mainframe back in the early 90’s. At that time we were looking for ways to deliver new technology and interfaces that would allow new technology, tools and applications to [be bought to] the mainframe. We also had a number of challenges to the IBM Development budget due to the drop in market perception of the mainframe through the late 1980’s – we considered and flirted with doing the WIN32 API but decided this was both technically and legally probably the wrong direction to go in, and soon after decided on the POSIX interfaces, file system etc. This latterly became part of the standards mantra at IBM and as the mainframe started to resurge, POSIX begat XPG4 which lead us to a SPEC1170 set of interfaces and included an implementation of the UNIX Distributed Computing Environment(DCE).

Some customers enthusiastically adopted these and the filesystem, others like yourselves were much more guarded in their approach. One area in which they were a hit was the delivery of new technology within IBM itself. We were able to port and reasonably quickly deliver OO technology in the form of SOM (SOMobjects for MVS) and later a DSOM implementation for OS/390; this in turn gave us Component Broker which is the forerunner of WebSphere. In late 1994 with IBM Hursley, we were able to port the then fledgling SUN OAK language, which by the time we’d ported it became Java; earlier the same year we ported an http server based on the then the same source that the Netscape server and later Apache [webserver] came from.

All these technologies/products/function were based in the OS/390, now z/OS UNIX System Services. Java and a number of key new technologies still run there today, as do the extensive connectors which allow the IBM HTTP Server and Java apps to connect to back-end DB2, CICS and IMS systems. In 1996, we even launched a box called the OS/390 Internet BonusPak which was a free bundle of software and manuals in a box to encourage its use(I still have a signed copy on the shelf behind me 🙂 ) – Mark Cathcart at WW5 in 1996I took an MVS 4.3 system, Internet Server running in USS and a fledgling Java implementation to the WWW5 Show in Paris in 1996, along with a connector which allowed Java applets to be streamed out of a back-end CICS transaction, and was mobed for the duration of the show.)

So the USS was first introduced with MVS/ESA 4.3(we did the first public demo at IBM ’93 in the UK) and a major part of MVS 5.2 and has been since. I personally architected an Internet banking system for NatWest in the UK that put 2x 9672 systems in a DB2 Sysplex outside their firewall in early 1999; it scaled from their initial 3000 users to over 994,000 without an outage of both systems at the same time, i.e. it was continuously available for 2-years; had no security exposures; had detailed logs of all file access, program invocation etc. as well as SMF records to allocate and track usage and bill back. Communication to the back-end backing systems was by MQ only with everything else locked-down and subject to hourly automated audits.

In the spring of 1998 I did a presentation to the S/390 Design Coucnil encouraging them to adopt XML. I understood the importance of this emerging technology, and most IBMers were able to quickly grasp it becuase of their vast exposure to DCF/SCRIPT either on VM/CMS or TSO/E. My proposal though backed with a solid technology which later became an industry standard, was a bit shaky. My proposal was that we do an end with screenscrapping by mapping 3270 into an XML Schema and building tools that allowed customers to exploit the wealth of mainframe applications through the web server using XML. Initially this idea had some useful direction and lead to a port of the XML Parser to the OS/390 UNIX Systems Services, while the parser survived and thrived, my idea died and was only much later adopted in part by the WebSphere Host Access Transformation Services.

The XML parser is a good example of a crossover service. It is available in LINUX on zSeries for LINUX apps to use; it is available in the z/OS USS for USS apps, and increasingly backend systems and z/OS languages and compilers now have access to a non-USS set of XML services.

So I can attest to USS working pretty well. By the spring of 1999, an optimistic but ultimately doomed effort to package up USS and a mainframe as a UNIX server with something called Auto-Unix had been implemented in our Boeblingen lab in Germany. The team that did this work and the skills they acquired, as well as the fact some were just co-ops from University, meant they had been exposed to the GNU toolset and the LINUX operating system. One thing lead to another and by mid-’99 we had a port of LINUX on S/390 hardware, we then spent the later part of ’99 and early 2000 reworking the strategy and getting adoption for LINUX on zSeries and more importantly, making LINUX a key part of the overall IBM strategy and a linchpin for our open standards drive.

I remember fondly, a very passionate discussion about this strategy at a wine festival in Germany in evening before we disclosed it to SAP in Waldorf. It was a seminal time for me and lead to me chaging my thinking completely, based on the response in the press[and media] and by customers in the subsequent eight-years, I wasn’t the only one.

So, what to make of this dual zSeries UNIX strategy ? Well, 1. LINUX is LINUX is LINUX. Its a complete implementation that can take re-compiled applications often without modification. While it picks up some huge benefits from running under zVM and on zSeries legendary hardware, it is still LINUX. As such it can only achieve the same scalability, performance and security of LINUX but running on zSeries. Once LINUX running in a single virtual machine has run out of performance or capacity, it can be replicated and an additional copy run on the same zVM system often for almost zero extra cost using up additional available capacity.

The UNIX System Services are NOT LINUX and are NOT UNIX. Applications built to run on UNIX often need small but important updates to run on z/OS. Sometimes you may have to do a major source code update in order to get the performance, scalability and throughput you’d expect from an application running on z/OS. On the other hand, unlike programs running on LINUX on zSeries, programs running in the USS are running on z/OS. With all its’ high availability characteristics, security, monitoring and management.

If your applications run on LINUX and need to be hosted in a secure, centralised, consolidated environment then LINUX on zSeries is one of the best. I would be remiss not to point out my current favourite, see this blog entry: http://ibmcorner.com/2007/02/15/linux-server-consolidation-a-go-go/

If on the other hand you have UNIX programmes, or UNIX skilled programmers and you want to port or run a specific application using z/OS; or perhaps write custom drivers to connect to specialised, highly available z/OS transaction server or application, USS would make a good place to start.

On 2nd thoughts, having written this much, do you mind if I do indeed post it, without your email, or your name as a blog entry? It makes a useful bookmark in the history of the mainframe.

If you have any other questions please feel free to fire away, I am though likely to forward them to friends who still work in the mainframe arena.

As a comeback, Dale asked:

these days are not all “UNIX” systems (especially considering market share) dependent on a base hardware/software platform or “tweaked” in some way. As a result, applications can not be easily moved from one system flavor to another? I am thinking of AIX, HP-UX, SCO-Unix, etc. So why is USS any less ‘UNIX’ than these other systems.

If I’m understanding, I think you are saying that one would have much more flexibility (less dependence on base hardware/software) with moving/porting LINUX applications in a “production” environment to another LINUX system. And I take it, scaling out a pSeries Linux application (with ‘production’ turnaround requirements) is much easier and cost effective.

My response was:

Yes of course you are right, all UNIX implementations have some differences, no matter how small. Even if they were the same, they wouldn’t be. Take for example SUN Niagara based Solaris systems, if you use the full threading capability of these boxes, despite the inherent poor performance of the single thread, the application run on another system, even if it compiled cleanly with no changes, is likely to perform very differently. In the case of USS they run in an EBCDIC environment which means that even 100% compatible API’s then you may have to change apps based on how they use data.

Yes, I’d agree with your interpretation of my advice, Linux does give you a greater degree of portability, although even it still depends package availability etc. note that I’m not saying my advice is to use Linux…. USS still has an important role to play, for example major vendor applications such as SAP exploit it for their products, its not just the IBM features and functions I listed.

Finally, since I’ll probably append this to the original response when I post it to the blog, not everything is as I’d have liked it. One of my big disappointments is that we didn’t continue to pursue and enhance the OpenEdition environment on VM/ESA, now z/VM. This was mostly an investment versus return decision, and not a technical one. I don’t think it would have made any difference to the porting and availability of Linux on zSeries, but it would have made z/VM a continuing vibrant platform for native applications and virtualization. While many new to zSeries now marvel at its’ virtualization capabilities, for me it was always the virtualization AND vibrant, flexible and dynamic personal application environment that was its real attraction. Lack of a viable USS on z/VM meant no JVM. Shame.

Funny then that for me when searching for “porting unix applications” on google returns the 1999 IBM Redbook “Porting UNIX Applications to OpenEdition for VM/ESA ” first.

Since this entry turned into a bit of a history, I thought I’d add a few names, as always I’m terrible remembering names so I’ll go back and look through my notebooks:

z/OS Unix Systems Services: Don Schmidt

LINUX for zSeries: Boas Betzler

IBM HTTP Server for MVS: Pat LiVecchi and John M Thompson

SOM, DSOM, CB for OS/390: Jeff Frey, Alan Little, Sunny Fulkerson, Garry Puchkoff

Java for MVS: Alan Webb, Mike Oliver

CICS Web enablement: Steve Wood, Susan Malika

DB2 Java enablement: Peggy Abelite/Zagelow

OS/390 XML Parser: Bill Carey

Tech Support: Geoff Hopkin, Cliff Laking, Paul Wanish

9 Responses to “Looking back, Unix System Services, Linux et al”


  1. 1 James Governor March 4, 2007 at 12:32 pm

    z/VM – indeed. it used to be too expensive to consider to take advantage of. that just changed with the new z/VM pricing. we may see your wish come true- as you know z is a long game..

  2. 2 Colin Renouf March 21, 2007 at 6:08 pm

    Mark

    Just thought you should know….

    Since I have got a complete stack now Kerberised with LDAP to join up AIX, WebSphere, Oracle, and other parts of the, I have been approached by our zOS team to see if they can apply the same principles to the z platform to ease their management burden and achieve single sign on using Windows Active Directory as the authenticator. Since Kerberos and LDAP started out as a “Unix-thing” this could be an interesting little project that may tie in to your comments above!

    Cheers mate

    Colin
    P.S. Try Herclues to play with zLinux. My personal belief is that the “Linux is Linux” ideal is likely to fall apart now that Microsoft and Novell/Suse are in partnership. Already the “standard” SELinux NSA designed security has gone in favour of the “easier” AppArmour. Only time will tell….

  3. 3 cathcam March 22, 2007 at 3:22 pm

    Thanks Colin, appreciate the comment. I’ve had a good work out with zLinux, albeit sometime ago, I was one of the co-authors of the first IBM Redbooks on zLinux, http://www.redbooks.ibm.com/abstracts/sg244987.html

    It’s my opinion that Linux will survive and thrive no matter comes out of the Microsoft/Novell partnership.

  4. 4 Colin Renouf March 23, 2007 at 10:27 am

    I completely agree that Linux will survive, but I suspect there may be two different and subtly incompatible brands: Suse and everyone else.

    Linux, particularly from Kernel 2.6 onwards, is a good OS. I do have concerns about the namespace in the driver model being too open.

    Is it worth going back to the OS “personalities” idea from the Taligent days? It may not have been successful then, but in many ways Apple are being successful with it now. So, what about a Linux/Intel emulation layer in a virtualised environment (I’m deliberately not being too specific) on System P/AIX? Apple have a new Mac, and old Mac, and a FreeBSD personality on Mac OS X and it happily supports most Linux code with a simple recompile – and it is all seamless. They then added Rosetta to allow a mix of PowerPC and Intel binaries to run seamlessly. What about using similar ideas in the enterprise space and coupling it with virtualisation?

    Cheers

    Colin

  5. 5 bloonsterific July 10, 2009 at 7:38 am

    Just wanted to tell you all know how much I appreciate your postings guys.
    Found you though google!

  6. 6 fax software August 12, 2010 at 1:31 am

    Instead of criticising write the variants.


  1. 1 Famous last words of Marius » This realy shows my age … the world of IBM mainframes Trackback on March 4, 2007 at 1:36 am
  2. 2 James Governor’s Monkchips » links for 2007-03-05 Trackback on March 5, 2007 at 5:43 pm
  3. 3 Unix System Services roll-out history » SDLC Blog Trackback on March 19, 2007 at 6:10 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




About & Contact

I'm Mark Cathcart, formally a Senior Distinguished Engineer, in Dells Software Group; before that Director of Systems Engineering in the Enterprise Solutions Group at Dell. Prior to that, I was IBM Distinguished Engineer and member of the IBM Academy of Technology. I'm an information technology optimist.


I was a member of the Linux Foundation Core Infrastructure Initiative Steering committee. Read more about it here.

Subscribe to updates via rss:

Feed Icon

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 998 other followers

Blog Stats

  • 83,308 hits

%d bloggers like this: