Archive for the 'opensource' Category

I left IBM in 2008, last week I said goodbye

I decided to post this over on my main blog as it was more to do with the people and community than about IBM. It contains some great references and links to content.

#HEARTBLEED was 5-years ago.

I was reading through my old handwritten tech notebooks this morning, search for some details on a Windows problem I know I’ve had before. I noticed an entry for March 28th, 2014 on the latest bug tracker list from Red Hat. One of the items on the list from the week before was the #Heartbleed bug in OpenSSL.

heartbleed-twoway-featured[1]

Image from synopsis.com

In less than a couple of weeks, Jim Zemlin from the Linux Foundation contacted John Hull in the open source team at Dell, who passed the call to me. I was happy to tell Jim we’d be happy to sign up, I got voice approval for the spending commitment and the job was done.

The Core Infrastructure Initiative (CII) was announced on April 24th, 2014. One of the first priorities was how to build a more solid base for funding and enabling open source developers. The first projects to receive funding were announced on April 26th, 2014 with remarkable speed.

Five years later I’m delighted to see Dell are still members, along with the major tech vendors, especially and unsurprisingly, Google. Google employees have made both substantial commitments to CII and open projects in general. I remember with great appreciation many of the contributions made by the tehn steering committee members, especially, but not limited to Ben Laurie and Bruce Schneier.

This blog, on synopsis.com, has a summary, entitled Heartbleed: OpenSSL vulnerability lives on. May 2, 2017.

My blog entries on Heartbleed and CII are here, here, and here.

There is still much to be concerned about. There are still many unpatched Apache HTTPD servers, especially versions 2.2.22 and 2.2.15 accessible on the Internet.

Remember, just because you don’t see software, it doesn’t mean it isn’t there.

Open Source redux

While I don’t update here much anymore that’s mostly because I’ve not been active in the general technology scene for the last 2.5 years following my departure from Dell and the resultant non-compete. I’m taking a few easy steps back now, I’ve reactivated my British Computer Society (BCS) Fellow membership and am hoping to participate in their Open Source Specialist Group meeting and AGM on October 25th.

MS-DOS Open Source

msdos-logo-150x150[1]Interestingly, Microsoft have announced they are re-open sourcing the code for MS-DOS 1.25 and 2.0 releases. Although never available outside of Microsoft or IBM in its entirety, there were certainly sections o the code floating around in the mid-1980’s. I was given the code for some drivers in 1984 by an IBM Systems Engineer, which I proceeded to hack and use as a starter for the 3270 driver I used for file transfer.

I’ve got a copy of the code released by Microsoft, and other the next 6-months am going to set about compiling it and working to get to work on a PC as a way to re-introduce myself to working in PC Assembler and the current state of compilers.

The Zowe Open Source Project

This was announced today at SHARE St Louis. A great new effort and opportunity to integrate open source technologies and applications into the IBM z/OS operating system. Zowe, as the article says, is

a framework of software services that offers industry standard REST APIs, API catalog, extensible command line interface and web-based UI framework

They’ve also put together the zowe,org community for architects, developers and designers to share best practices. It’s not clear what the legal relationship is between the open mainframe project and zowe, but zowe is listed as a project, so that’s great news in terms of strategy and direction. As of writing, the open mainframe zowe project web page has the best detail on the project.

Zowe appears to be a collaboration between IBM and a number of companies, including Rocket Software. Rocket has a broad portfolio of software and systems that integrate with IBM Systems, they also have my friend, former colleague and sparing partner at IBM, Jim Porell on staff.

Open Distributed Challenges – Words Matter

I had an interesting exchange with Dez Blanchfield from Australia on twitter recently. At the time, based on his tweets, I assume Dez was an IBM employee. He isn’t and although our paths crossed briefly at the company in 2007, as far as I’m aware we never met.

The subject was open vs open source. Any longtime readers will know that’s part of what drove me to join IBM in 1986, to push back on the closing of doors, and help knock down walls in IBM openness.

At the end of our twitter exchange, the first 3-tweets are included above, I promised to track down one of my earlier papers. As far as I recall, and without going through piles of hard copy paper in storage, this one was formally published by IBM US using a similar name, and pretty much identical content, probably in the Spring 0f ’96.

It is still important to differentiate between de jure and de facto standards. Open Source creates new de facto standards every day, through wide adoption and implementation using that open source. While systems ,ove much more quickly these days, at Internet speed, there is still a robust need to de jure standards. Those that are legally, internationally and commonly recognised, whether or not they were first implemented through open source. Most technology standards these days are as that’s the best way to get them through standards organizations.

The PDF presented here is original, unedited, just converted to PDF from Lotus Word Pro.

Lotus Word Pro, and it’s predecessor, Ami Pro, are great examples of de facto standards, especially inside IBM. Following the rise of Microsoft Word and MS Office, Lotus products on the desktop effectively disappeared. Since even inside IBM, the Lotus source code was never available, not only were the products only a de facto standard, they were never open source. While in the post Lotus desktop software period considerable effort has been put into reverse engineer the file formats , and some free and chargeable convertors almost all of them can recover the text, most do a poor job or formatting.

For that reason, I bought a used IBM Thinkpad T42 with Windows XP; Lotus Smartsuite and still have a licensed copy of Adobe Acrobat to create PDF’s. Words matter, open source, open, and open standards are all great. As always, understand the limitations of each.

There are a load of my newer white papers in the ‘wayback’ machine, if you have any problems finding them, let me know, I’ll jump start the Thinkpad T42.

Remembering the dawn of the open source movement

and this isn’t it.

attwood statistics 1975

Me re-booting an IBM System 360/40 in 1975

When I first started in IT in 1974 or as it was called back then, data processing, open source was the only thing. People were already depending on it, and defending their right to access source code.

I’m delighted with the number and breadth of formal organizations that have grown-up around “open source”. They are a great thing. Strength comes in numbers, as does recognition and bargaining power. Congratulations to the Open Source Initiative and everything they’ve achieved in their 20-years.

I understand the difference between closed source, (restrictive) licensed source code, free source, open source etc. The point here isn’t to argue one over the other, but to merely illustrate the lineage that has led to where we are today.

Perhaps one of the more significant steps in the modern open source movement was the creation in 2000 of the Open Source Development Labs, (OSDL) which in 2007 merged with the Free Standards Group (FSG) to become the Linux Foundation. But of course source code didn’t start there.

Some people feel that the source code fissure was opened when  Linus Torvalds released his Linux operating system in 1991 as open source; while Linus and many others think the work by Richard Stallman on the GNU Toolset and GNU License started in 1983, was the first step. Stallman’s determined advocacy for source code rights and source access certainly was a big contributor to where open source is today.

But it started way before Stallman. Open source can not only trace its roots to two of the industries behemoths, IBM and AT&T, but the original advocacy came from them too. Back in the early 1960’s, open source was the only thing. There wasn’t a software industry per se until the US Government invoked its’ antitrust law against IBM and AT&T, eventually forcing them, among other things, to unbundle their software and make it separately available as well as many other related conditions.

’69 is the beginning, not the end

The U.S. vs.I.B.M. antitrust case started in 1969, with trial commencing in 1975(1). The case was specifically about IBM blocking competitive hardware makers getting access and customers being able to run competitive systems, primarily S/360 architecture, using IBM Software.

In the years leading up to 1969, customers had become increasingly frustrated, and angry at IBM’s policy to tie it’s software to its hardware. Since all the software at that time was source code available, what that really meant was a business HAD to have one IBM computer to get the source code, it could then purchase an IBM plug-compatible manufacturers (PCM) computer(2) and compile the source code with the manufacturers Assembler and tools, then run the binaries on the PCM systems.

IBM made this increasingly harder as the PCM systems became more competitive. Often large previously IBM only systems users who would have, 2, 4, sometimes even 6 IBM S/360 systems, costing tens of millions of dollars, would buy a single PCM computer. The IBM on-site systems engineers (SE) could see the struggles of the customer, and along with the customers themselves, started to push back against the policy. The SE job was made harder the more their hands were tied, and the more restrictions that were put on the source code.

To SHARE or not to?

For the customers in the US, one of their major user groups, SHARE had
a vast experience in source code distribution, it’s user created content, tools tapes were legend, what most never knew, is that back in 1959, with General Motors, SHARE had its own IBM mainframe (709) operating system, the SHARE Operating System (SOS).

At that time there was formal support offerings of on-site SE’s that would work on problems and defects in SOS. But by 1962, IBM had introduced it’s own S/7090 Operating System, which was both incompatible with SOS, and also at that time IBM withdrew support by it’s SE and Program Support Representatives (PSR’s) to work on SOS.

1965 is where to the best of my knowledge is when the open source code movement, as we know it today, started

To my knowledge, that’s where the open source code movement, as we know it today, started. Stallman’s experience with a printer driver mirrors exactly what had happened some 20-years before. The removal of source code, the inability to build working modifications to support a business initiative, using hardware and software ostentatiously already owned by the customer.

IBM made it increasingly harder to get the source code, until the antitrust case. By that time, many of IBMs customers had created and depended on small, and large modifications to IBM source code.

Antitrust outcomes

Computerworld - IBM OCOBy the mid-70’s, once of the results of years of litigation, and consent decrees in the United States, IBM had been required to unbundle its software, and make it available separately. Initially it was chargeable to customers who wanted to run it on PCM, non-IBM systems, but overtime as new releases and new function appeared, even customers with IBM systems saw a charge appear, especially as Field Developed Programs, moved to full Program Products and so on. In a bid to stop competing products, and user group offerings being developed from their products, this meant the IBM Products were increasingly supplied object-code-only (OCO). This became a a formal policy in 1983.

I’ve kept the press cutting from ComputerWorld(March 1985) shown above since my days at Chemical Bank in New York. It pretty much sums-up what was going on at the time, OCO and users and user groups fighting back against IBM.

What this also did is it gave life to the formal software market, companies were now used to paying for their software, we’ve never looked back. In the time since those days, software with source code available has continued to flourish. With each new twist and evolution of technology, open source thrives, finds it’s own place, sometimes a dominant position, sometimes subservient, in the background.

The times in the late 1950’s and 60’s were the dawn of open source. If users, programmers, researchers and scientists had not fought for their rights then, it is hard to know where the software industry would be now.

Footnotes

(1) The PCM industry had itself come about as a result of a 1956 antitrust case and the consent decree that followed.

(2) The 1969 antitrust case was eventually abandoned in 1982.

Do you own the device you just bought?


Professor of Law, Washington and Lee University, has a great blog post that echoes exactly the same sentiments I heard Richard Stallman explain his original drive for open source, way back in the 1980’s.

Fairfield argues that we don’t own the devices we buy, we are merely buying a one-time license to the software within them. He makes a great case. It’s worth the read.

One key reason we don’t control our devices is that the companies that make them seem to think – and definitely act like – they still own them, even after we’ve bought them. A person may purchase a nice-looking box full of electronics that can function as a smartphone, the corporate argument goes, but they buy a license only to use the software inside. The companies say they still own the software, and because they own it, they can control it. It’s as if a car dealer sold a car, but claimed ownership of the motor.

My favorite counter-example of this is the Logitech Squeezebox network music player system I use.  Originally created by Slim Devices, as far back as 2000, with their first music player launched in 2001. Slim Devices were acquired by Logitech in 2006, who then abandoned the product line in 2012.

I started using Logitech Squeezebox in 2008, first by buying a Squeezebox Boom, then a Radio, another Boom, a Touch and have subsequently bought used Duet, and for my main living room, the audiophile quality Transporter.

While there are virtually no new client/players, there is a thriving client base built around the Raspberry Pi hardware with both client software builds and add-on audio hardware, as well as server builds to use the Pi. I’ve hacked some temporary preferences into the code to solve minor problems, but by far the most impressive enhancements to the long abandoned, official, server codebase are the extensions to keep up with changes in streaming services like the BBC iPlayer radio, Spotify, DSD play and streaming and many more enhancements. For any normal, closed source platform any one of these enhancements would likely have been impossible, and for many users made the hardware redundant.

The best place to start in the Squeezebox world is over on the forums, hosted, of course, at http://forums.slimdevices.com/

When my 1-month Ring (video) doorbell failed. It was all I could do to get Ring to respond. I spent nearly 4-hours on the phone with tech support. Not only did I have no control, the doorbell had stopped talking to their service, but they couldn’t really help. After the second session with support, I just said “look I’m done can you send a replacement?” – The tech support agent agreed they would, but 10-days later I was still waiting for even a shipping notice, much less a replacement. While the door bell worked as a door bell, none of the services, motion detection, door bell rings were any good as their services were unavailable to my door bell.

You don’t have to give up control when you buy a new device. You do own the skeleton of the hardware, buy you’ll have to make informed choices, and probably will give up control, if you want to own the soul of the machine, it’s software.

The app hell of the future

Just over 5-years ago, in April 2011, I wrote this post after having a fairly interesting exchange with my then boss, Michael Dell, and George Conoly, co-founder and CEO of Forrester Research. I’m guessing in the long term, the disagreement, and semi-public dissension shut some doors in front of me.

Fast forward 5-years, and we are getting the equivalent of a do-over as the Internet of Things and “bots” become the next big thing. This arrived in my email the other day:

This year, MobileBeat is diving deep into the new paradigm that’s rocking the mobile world. It’s the big shift away from our love affair with apps to AI, messaging, and bots – and is poised to transform the mobile ecosystem.

Yes, it’s the emperor’s new clothes of software over again. Marketing lead software always does this, over imagines what’s possible, under estimates the issues with building in and then the fast fail product methodology kicks-in. So, bots will be the next bloatware, becoming a security attack front. Too much code, forced-fit into micro-controllers. The ecosystem driven solely by the need to make money. Instead of tiny pieces of firmware that have a single job, wax-on, wax-off, they will become dumping ground for lots of short-term fixes, that never go away.

Screenshot_20160524-113359Meanwhile, the app hell of today continues. My phone apps update all the time, mostly with no noticeable new function; I’m required to register with loads of different “app stores” each one a walled garden with few published rules, no oversight, and little transparency. The only real source of trusted apps is github and the like where you can at least scan the source code.IMG_20160504_074211

IMG_20160504_081201When these apps update, it doesn’t always go well. See this picture of my Garmin Fenix 3, a classic walled garden, my phone starts to update at 8:10 a.m., and when it’s done, my watch says it’s now 7:11 a.m.

IMG_20160111_074518Over on my Samsung Smart TV, I switch it from monitor to Smart TV mode and get this… it never ends. Nothing resolves it accept disconnecting the power supply. It recovered OK but this is hardly a good user experience.

Yeah, I have a lot of smart home stuff,  but little or none of it is immune to the app upgrade death spiral; each app upgrade taking the device nearer to obsolescence because there isn’t enough memory, storage or the processor isn’t fast enough to include the bloated functions marketing thinks it needs.

If the IoT and message bots are really the future, then software engineers need to stand up and be counted. Design small, tight reentrant code. Document the interfaces, publish the source and instead of continuously being pushed to deliver more and more function, push back, software has got to become engineering and not a form of story telling.

YesToUninstallAnUpdate[1]

Retired Until Further Notice

RUFN. I can’t remember where I first saw this, I think on an ex-colleagues linked-in status(*1). Back in September I declared I was done with cube life and it didn’t take long before it was time to part company with Dell.

I’m at an important crossroads, starting to pack up my Austin home, and move to a new house my partner, Kate, and I are building just south east of Boulder CO. Kate is already living in Boulder, where we are partners in Boulder Bodyworker.

So it seemed like an appropriate time to take some time out, and start an exciting new phase of life for me. I’ll be keeping busy, while I don’t have any active movie or music projects at the moment, I am behind on working on a project for Tri Equal and also a member of the advisory board  of the Professional Triathlon Union and continuing generally as an activist in the triathlon community.

I’m available for consulting work in the new year, especially for small to medium sized businesses that want to get an insight or review of their technology strategy; a perspective and advice on working with open source; data center operations.

Otherwise I’ll post here as appropriate and see how things develop next year. Merry Christmas and a Happy New Year

 

*1. Yeah I’m aware of the slang usage.

O’Reilly Webcast – Extending Cassandra for OLAP

oreilly doradusColleague Randy Guck, who leads our open source Doradus project, recent gave an O’Reilly Webcast on the project and using Doradus to extend Cassandra for high performance analytics.

The discussion on how Doradus leverages Cassandra, its data model and query language, the internal architecture and the concept of storage services gave in-depth background to then understand the Doradus OLAP service and how it provides near real-time data warehousing.

Randys’ slides and webcast can be fund here. It does need registration, but is well worth the effort. The webcast was sponsored by Dell, which was entirely coincidental, since it was for a Hadoop services offering. Doradus offers some interesting ways to extend and use Cassandra and Randy covers most of them in the webcast. The key point is, that Doradus is an open source project, use and source code are free. Details on Doradus are in this blog entry.

OpenSSL and the Linux Foundation

Former colleague and noted open source advocate Simon Phipps recently reblogged to his webmink blog a piece that was originally written for meshedinsights.com

I committed Dell to support the Linux Foundation Converged Infrastructure Initiative (CII) and attended a recent day long board meeting with other members to discuss next steps. I’m sure you understand Simon, but for the benefit of readers here are just two important clarifications.

By joining the Linux Foundation CII initiative, your company can contribute to helping fund developers of OpenSSL and similar technologies directly through Linux Foundation Fellowships. This is in effect the same as you(Simon) are suggesting, having companies hire experts . The big difference is, the Linux Foundation helps the developers stay independent and removes them from the current need to fund their work through the (for profit) OpenSSL Software Foundation (OSF). They also remain independent of a large company controlling interest.

Any expansion of the OpenSSL team depends on the team itself being willing and able to grow the team. We need to be mindful of Brooks mythical man month. Having experts outside the team producing fixes and updates faster than they can be consumed(reviewed, tested, verified, packaged and shipped) just creates a fork, if not adopted by the core.

I’m hopeful that this approach will pay off. The team need to produce at least an abstract roadmap for bug fix adoption, code cleanup and features, and I look forwarding to seeing this. The Linux Foundation CII initiative is not limited to OpenSSL, but that is clearly the first item on the list.

Open Source @ Dell – Doradus

I’m delighted to announce that last week Dell Software group made available it’s first major open source project, Doradus.

Doradus is the next and biggest release so far from the software group at Dell and it joins Blockade, discussed in this blog. Through 2014, I hope to be in a position to announce at least a couple more big projects, and numerous smaller ones. We are pulling together a coherent approach to this, as well as a number of smaller tools.

What is Doradus?

Doradus is a set of tooling that started out ~2.5 years ago and is, and has been used by a number of our Dell software products. It has not been available as a product itself. Doradus provides a REST API on top of the Cassandra NoSQL database, adding a number of high level features. As a pure Java service it simplifies and extends NoSQL database functionality with a graph-based data model with bi-directional relationships and full referential integrity.

Included are a powerful query language supporting full text and statistical queries; Automatic data aging;  and Two storage services that target specific application types. An  OLAP service provides ultra-dense storage and fast analytic queries. There is a client library that allows Java clients to use POJOs to access Doradus DBs. It scales horizontally with Cassandra to provide NoSQL benefits of elasticity, replication, fault-tolerance, low cost, etc.

What was open-sourced?

The Doradus components included in the OSS offering are:

  • doradus-server: Source code and config files for the server.
  • doradus-client: Source code and config files for the Java client library.
  • doradus-common: Source code for doradus-common.jar, used by both the client and server modules.
  • docs: PDF versions of the main Doradus documentation. The build scripts in the root directory also build Java docs for the client library in the folder ./doradus-client/docs.

These components are released with the Apache License 2.0. Currently, we are working through the legal issues on enhancements and contributions, and will add an Apache based CLA to encourage larger contributions. In the interim we are happy to accept bug fixes for inclusion in the next code base rev. We are also looking to add the regression test suite that we use for continuous integration build integrity.

Where can I get it?

Doradus source code, documentation, and build scripts are available here: https://github.com/dell-oss/Doradus . You can use any Git client to download the files, or click the Download ZIP button to get everything as one .zip file. The root directory has both Ant and Maven build scripts, which download dependent jar files and build the binaries. In the near future, we will post pre-built source code, doc, and binary bundles on Maven Central to simplify downloading and installing.

What is dell-oss?

One of the things we’ll be doing this year is pulling together our open source projects and contributions, to make them easier to find, and to simplify for the Dell teams that will be contributing OSS projects. Personally, I’d like to also include a section where we store copies of our incoming and outgoing licenses, templates, and completed licenses. At least as of now we’ll be doing that through dell-oss, with Ant and Maven as needed. More detail on this when we make our next project announcement.

Congratulations to Randy Guck, James Bumgardner who made the OSS effort happen, also to the other Doradus developers.

More on OpenSSL, Heartbeat

I don’t propose to become an expert on OpenSSL, much less the greater security field, but I know people who are. My role in the Linux Foundation Core Infrastructure Initiative was to help Dell recognize how we can support a key industry technology, and at least give Dell the ability to have input on what comes next.

Our SonicWall team have many experts. They’ve published a great blog both on  their product positioning and use in relation to Heartbleed and vulnerabilities, and Network Security product manager Dmitriy Ayrapetov raises the question, in a world of mostly TCP traffic, are TLS Heartbeats even necessary?

The Dell SecureWorks Counter Threat Unit™ (CTU) have a blog on malware arising out of and exploiting the heartbleed vulnerability. Another great Dell resource well worth following for those with an interest in security.

Core Infrastructure Initiative (OpenSSL)

I’m pleased to announce that Dell with be a joining the Linux Foundation and a number of key industry partners in establishing the Core Infrastructure Initiative(CII). This is another open source initiative, and I’m glad to have have played my part in pushing through the approval. I mentioned in my February blog, and we continue to work on three other, I think significant initiatives.

CII is a new project to fund and support critical elements of the global information infrastructure. The Core Infrastructure Initiative enables technology companies to collaboratively identify and fund open source projects that are in need of assistance, while allowing the developers to continue their work under the community norms that have made open source so successful.

The first project under consideration to receive funds from the Initiative will be OpenSSL, which could receive fellowship funding for key developers as well as other resources to assist the project in improving its security, enabling outside reviews, and improving responsiveness to patch requests.

You can read the full Linux Foundation news release here and the New York Times already has a blog here.

Growing software influence and Dell

A few things have happened in the last couple of months that show the growing influence and maturity of the software team at Dell, and it’s been on my backlog to write up as a blog post.

DMTF VP of Regional Chapters

Yinghua Qin, the Senior Software Manager in our Zuhai China laboratory has been accepted as the new VP of Regional Chapters at the DMTF. This is an outstanding opportunity for Yinghua, who leads the Foglight and a number of software engineering projects, as well as serves as the local liaison to Sun Yat-sen University(SYSU) school mobile engineering (SMIE). Yinghua reports to the Foglight lead architect Geoff Vona.

Dell actually has at various stages in the past been very proactive with the DMTF. Current board chair, Winston Bumpus, was formally a Dell employee; My ESG colleague Jon Haas has been a major contributor to a number of standards. I for one am looking forward to the increased cooperation that working in international standards can bring.

Open Source Project

The Dell Cloud Manager product development team have open sourced their blockade test tool. Blockade is a utility for testing network failures and partitions in distributed applications. Blockade uses Docker containers to run application processes and manages the network from the host system to create various failure scenarios.

It’s a small step, but congratulations to Tim Freeman and the team for navigating through the process to produce the first new open source development project from the Dell Software Group team.

Angular giveback

A number of our development teams are using Angular.js. Once again after an original approach in November by Sara Cowles from the Dell Cloud Manager team stepped forward and asked the right questions, after checking with other teams, I was happy to sign the Google CLA to fax back to google.

Yocto – Embedded Linux and Beyond

Congratulations also go to Mikey Brown from Dells’ Enterprise Systems Group(ESG). Mikey has picked up the mantle of a project I was a big supporter of, when I was in ESG, Yocto. After doing a great job getting a couple of our embedded Linux offering back on track using Yocto, and the build infrastructure around. Mickey has re-connected with the Yocto team.

Each of these on their own are small steps, but these plus a number of other things going on give me a good feeling things are heading in the right direction. I’ll get to go have another facsinating time hearing from students about how things look from their side of the technology field when I head over to Texas A&M University(Insert “GO AGGIES” here!) to address class 481 on 2/25.


About & Contact

I'm Mark Cathcart, formally a Senior Distinguished Engineer, in Dells Software Group; before that Director of Systems Engineering in the Enterprise Solutions Group at Dell. Prior to that, I was IBM Distinguished Engineer and member of the IBM Academy of Technology. I am a Fellow of the British Computer Society (bsc.org) I'm an information technology optimist.


I was a member of the Linux Foundation Core Infrastructure Initiative Steering committee. Read more about it here.

Subscribe to updates via rss:

Feed Icon

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,066 other subscribers

Blog Stats

  • 90,343 hits