Open-sourcing the Apple

A hacker reviews the beta release of Mac OS X -- and dreams of toppling Microsoft.

Published November 17, 2000 8:30PM (EST)

If you travel in geek circles, by now you have no doubt heard about Apple's beta release of OS X, a wholly new operating system for the Macintosh. That's especially true if you are a member of that subset of geeks who closely follow open-source software issues.

OS X is a much anticipated amalgam of the Mach 3.0 microkernel from Carnegie-Mellon University, and FreeBSD 3.2, a more traditional open-source Unix-compatible operating system from the FreeBSD Project. But knowing that OS X is a microkernel wrapped up in a Unix OS, which is in turn wrapped up in a whole new layer of graphical user interface (GUI) technology, doesn't tell the whole story. Is OS X just another fancy GUI-based operating desktop system like Windows or is it a more industrially useful server-centric operating system like FreeBSD or Linux-based OS's? Crafting user interfaces is Apple's widely acknowledged forte; FreeBSD technology is known to power major Internet sites like Yahoo and Sony Japan. So which is it?

Or is it both? It is possible for one operating system to satisfy both the needs of someone like myself, a FreeBSD developer who expects a lot of power and flexibility from an operating system, and the average user who just wants to point and click?

If the answer to that question is yes, then the arrival of Mac OS X could augur some significant changes in the operating system marketplace. A powerful OS that runs popular applications would represent a Unix that has finally grown up. And it would present us with a truly interesting question: Should Microsoft be worried? I say yes, because Mac OS X can potentially challenge Windows both in usability and in industrial reliability; but, no, because Apple's slice of the market is still too small, and Microsoft's sway with developers and independent software vendors is too high.

Apple's chances would increase greatly if instead of merely incorporating portions of an open-source operating system in Mac OS X, the company fully committed to the open-source software development model and freed all of its OS source code. Being truly open would allow Apple the ability to spread its technology in an almost viral fashion to new markets, with an army of volunteers doing the kinds of hardcore programming work that would enable the Macintosh operating system to work on multiple hardware systems. Apple plus the open-source community could challenge Microsoft.

The rest of this essay is divided into two parts. Readers who are unafraid of a geekily technical (and Unix-heavy) appreciation of OS X can go right on to the next section. Readers who would rather cut to the chase for a consideration of the potential implications of OS X for the software industry should go straight to the final section.

The first thing I noticed upon initially booting up OS X is that it didn't go out of its way to advertise the fact that there was a Unix-compatible operating system under the hood. Unlike its FreeBSD, Linux or Solaris cousins, the boot-time messages that tell you exactly what is going on as the operating system takes full control of the computer were confined to a small pop-up dialog that announced the various major stages of the boot process and marked its overall progress with a colorful status bar. For the intended audience this was undoubtedly the right thing to do, but Unix purists like me will still miss seeing the device probes and other information that could conceivably be of use when debugging the system (which all Unix die-hards tend to do at one point or another).

After the system was fully up, a login-requester was displayed and I was able to log in using the administrator user name and password, which were requested during the installation of OS X. Once logged in, I was confronted by a standard icon dock at the bottom of the screen with a set of icons happily bouncing up and down as the applications they represented launched themselves. That's one of the first things I needed to get used to with OS X -- everything is very colorful and more than a little animated at times. I could live with that, but horrors, where was the terminal application -- that plain and simple window with command-line access to the raw power of the system?! Not to worry, Apple had simply hidden it a few clicks away. Inside the Utilities folder I found a terminal application and just a couple of clicks more presented me with a terminal window running tcsh, a version of the C shell that supports filename completion and other useful extensions that should be familiar to just about anyone who's used Unix before. So far, so good!

Once I had access to a command line, OS X started looking a lot more like a Unix system. A df command showed which file systems were mounted and I could traverse the file systems with all the standard Unix commands like find, ls, cd, etc. There are even copies of emacs 20.7 and Perl 5.6.0 installed by default, applications more typical of a "power user" configuration than what I'd expect to find on a standard out-of-box Unix installation and something of a pleasant surprise.

Unfortunately, the converse was also true and the next thing I noticed was that the C/C++ compiler suite was missing. For me and most developers, this could be rectified by simply installing the Developer package from the OS X Developer CD-ROM, an option in the beta for people who are part of Apple's Developer program. But this really should be a standard part of the final OS X release product and not something only "developers" can get or, worse, have to pay for. The compiler tool-chain is simply too much a de facto part of any system these days to make it something that can only be gotten by jumping through extra hoops, and I was a little disappointed to see that it was even unbundled by Apple in the first place.

Of course, the proof that I was working with a real operating system would be to compile something -- to get some of my favorite Unix-ish software running on this system. So I promptly fetched the source code to the GNU Project's bash 2.04, my favorite shell and one that didn't happen to be included with OS X. I unpacked it without any trouble. (Let's hear it for GNU tar, gzip, cpio and pax being standard components!)

I did run into some problems here. Once bash's configuration script was told it was on a "powerpc-bsd" machine, the build proceeded normally until the Makefile attempted to link in the GNU malloc library. If you are not Unix-familiar you probably don't want to know the details, but suffice to say, the attempted link was a bad idea and I had to do some minor tinkering to get a functional bash shell.

But as a portability benchmark -- a criterion with which to judge how easy it is to get foreign software running in Mac OS X -- this was certainly not bad at all and I had much the same good results with TCL, another popular open-source application. Porting Unix software to OS X is clearly far less work than trying to port it to Windows 2000 and with OS X providing such a high degree of Unix-compatibility, something like the FreeBSD ports collection (which highly automates the process) would make the third-party software situation pretty close to ideal.

The next thing I did was telnet into the machine to administer it from a desktop environment I was more familiar with. This is something else OS X does right out of the box (under control of the "Sharing" preferences). It even announces itself as "Darwin/BSD" in the login banner, letting one know you'll be in for a fairly typical BSD experience if you come in that way.

Of course, telnet is somewhat insecure, so I immediately looked for a copy of sshd (the secure shell daemon) to start up so I could do my telneting in complete security. Unlike the developer pre-releases that preceded it, this release of OS X included sshd. But this was also where OS X started to look just a little different from FreeBSD, which starts sshd by default. OS X doesn't have any obvious way of starting sshd at system startup time and after tinkering with various files for a while I gave up; clearly this is an area where OS X is significantly divergent and the documentation is still unclear.

Another area of confusion for me was the editing of user account data. On a BSD machine, for example, one can run the "chsh" command to set a user's default shell to be something else (like bash). Using chsh is also possible on OS X, the only problem is that it doesn't work. Only editing that information with the NetInfoManager made it stick and this is one of those areas where the OS X way and the traditional Unix way are both present, but also directly in conflict. Getting services like NIS or Kerberos to work correctly with OS X is something I didn't even want to try after looking into it for a while and I suspect that this will come up as a problem for anyone trying to assimilate OS X into existing heterogeneous Unix networks.

Apple's support for "resource forks" (roughly described as extended file attributes in the Unix world) is also a feature anyone wishing to use OS X largely as a Unix server machine will want to avoid given that resource fork information cannot be backed up and restored easily using any of the standard Unix backup tools.

Still, this is a beta release and it was impressive just how much "Unix stuff" did work exactly as I'd expected. One can easily log into the machine remotely or use a full suite of network tools such as ftp, ssh and NFS to access files on other machines, and the built-in Apache Web server can also be easily launched at the click of a button in the System Preferences tool. Considering just how much more network-capable OS X is out of the box when compared to a comparable Windows installation, it's almost embarrassing to complain about it.

Once I got used to the windows leaping around and contorting themselves on their way to and from the icon dock, the GUI front-end grew on me and didn't send me leaping for the terminal icon every time I wanted to interact with the machine. It's not exactly designed with the die-hard Unix user in mind, but the approach Apple has taken to solving all the classic interface problems is fairly intuitive and it didn't take much playing with the desktop before it at least wasn't getting significantly in my way, which is more than I can say for most GUIs.

Far more important to me were the Mac OS 9 "classic" compatibility features, which gave me the ability to take retail software like Adobe Photoshop for Mac OS 9 and run it out of the box, something I did on the second day I had the system installed. A classic consumer application like Photoshop for the Mac and a Unix standby like emacs running side-by-side was certainly not something I ever expected to see.

It's one of the most exciting aspects of the system: Mac OS X offers the prospect of a "real OS" with "real applications," not just the big, slow, late and far more expensive versions of Windows software most Unix users have sadly become accustomed to.

With multiple millions of Macintosh users to point to, Apple also stands to have little trouble convincing the major software vendors and developers to make versions of their software for OS X as well. We'll have to wait and see how it all plays out, but it looks very promising.

So what does this mean for Microsoft's world software hegemony?

Microsoft achieved its dominant position by cracking the traditional chicken-and-egg problem that faces any manufacturer of a new operating system: How do you simultaneously attract users to an operating system before it has lots of popular applications and convince software developers to create those applications before there is a significant user base?

Microsoft was pretty smart about this and realized that in order to get those initial software developers it had to do several significant things. For one, it had to win over the engineering managers in a position to make "cost decisions" about which software ports to do. This was largely done by designing and implementing a set of APIs -- application programming interfaces -- to deal with almost every immediate application requirement, from data exchange to database connectivity. An API, simply put, is a standard set of code that connects an application to an operating system. Microsoft aggressively pushed its WIN32 API as something that independent developers and vendors could not live without if they wanted to get their software to market quickly.

To convince the engineers, especially those fresh out of school, a whole host of Visual This and Visual That tools were also created and the advantages of Microsoft's Integrated Development Environments [IDEs] were widely touted as the best invention since coffee for programmers.

It didn't really matter so much that some of those APIs were hardly best-of-breed designs or that the IDEs and their associated documentation often fell far short of claims. It was enough to be seen as trying to solve those problems and Microsoft eventually got enough significant applications for Windows that it became an operating system users couldn't live without, thus closing the loop with Microsoft sitting very comfortably in the center and able to launch an aggressive application development program of its own (Microsoft Office, etc.) in order to bring in an even bigger slice of the pie it had, in effect, created.

What does this have to do with Apple? More than you'd think, really. For many years now, Microsoft has been the Goliath everyone complains about, but no one sees any effective way of dealing with. Not to mention the deceased pile of would-be Davids lying at its feet that serve as something of a deterrent to others. The open-source world, on the other hand, has taken the approach that if enough ants go after Goliath with their painful stingers, you don't need a David.

The real truth is somewhere in the middle. As effective as the open-source community has been at giving Microsoft something to worry about, it needs a David to provide both a point of focus for the overall battle and the kind of heavy lifting that simply can't be accomplished by a distributed mass of ants.

This is where I think Apple could play a very significant role -- if it only saw that the various programmers' APIs it has created for components of OS X represent more than just a pretty face and a something for developers and vendors to latch on to. They also represent the first credible beginnings of a challenge to the almost ubiquitous WIN32 API. It is through control of WIN32 that Microsoft has maintained its lock on most of the world's applications software, after all, and thus on the computing industry as a whole.

I believe that the open-source community, for all of its good points, lacks enough leverage to push its own GUIs and integrated desktop environments, such as GNOME or KDE, into the mainstream far enough that we're ever likely to see Adobe Photoshop for KDE or Microsoft Internet Explorer for GNOME (nevermind whether or not you'd want to). The general lack of independent software developer and vendor confidence in either platform leads only to deadlock in the chicken-and-egg scenario described earlier. I don't mean any insult to the KDE or GNOME people by this since they've done a lot of amazing work in a very short period of time. I simply think it's not inaccurate to say that the situation doesn't look too promising for widespread Fortune 500 acceptance of either technology soon.

So, into this situation comes Apple with its relatively large installed user base and some promising new user interface technology. The company has managed to focus a dedicated, full-time group of very smart people long enough to produce some significant results. What comes next depends largely on how bold a game Apple and its technology partners are willing to play.

One game, of course, would be to see how many copies of OS X it can sell to the existing Macintosh user base while striving as aggressively as possible to ensure that new computer users choose Macintosh hardware and software over the latest offerings to come out of the hugely successful (or just plain huge) PC market. Given the tremendous economies of scale and rate of innovation enjoyed by the PC market, not to mention the fact that Microsoft is firmly allied with it, that would be something of a tough battle, to put it mildly.

Another game Apple could play, and this assumes a very bold and aggressively forward-thinking Apple indeed, would be to open-source almost the entirety of OS X. The decision to release something of that magnitude would obviously also not be an easy one for Apple's upper management to swallow. It would first have to see this not as a "sacrifice play," but rather as the first significant move in a much larger fight for a technology and market leadership position. It's certainly easier to measure short-term success by the number of shrink-wrapped software boxes shipped and ultimately settle for a position somewhere behind Microsoft, but that's been the same game everyone has been playing for quite a while and one can only hope that Apple has more ambitious and strategic goals.

That is not to say that having source code available on the Internet implies that the market for prebuilt, tested and supported versions of a product will completely go away -- that's a commonly held fallacy. Other open-source operating system companies have sold hundreds of thousands of copies of software that is also freely available in source form on the Net. A lot of people are still consistently willing to pay for a certain level of assurance that they're getting their software from the experts. Who would know OS X better than Apple?

More importantly, openness would give Apple access to the open-source community of developers. It would even give the company entree into the PC world -- there's an army of volunteers doing the kinds of device driver and hardware support work that Apple would have a hard time trying to do all by itself. Having a credible GUI alternative for open-source operating systems might also, finally, move the open-source world past trying to reinvent the desktop wheel and on to writing the kinds of developer productivity software and "generally useful stuff" that a large user base with access to source code can provide in almost unending quantity.

To be certain, there will exist a cadre of OS X developers doing nifty Aqua applications whether the sources are released or not, but that group won't include so many of the talented BSD and Linux people who are backing other technology simply because it's free, both ideologically and monetarily, and the importance of that community should never be underestimated.

Through all of this, Apple would still be at the center of the resulting cyclone since it would have so much "moral authority" over the APIs and would be at the center of so much ongoing innovation that any erstwhile competitor could only ignore Apple at its own peril. It just simply wouldn't make any sense to leap off the biggest bandwagon in town. Apple would be able to easily leverage its center position in any number of ways by having some of the best people and always enough lead-time on emerging market opportunities to grab some of the best for itself. Apple would also have plenty of room for a Microsoft-like transition to writing higher margin application suites (which would NOT be free) while also continuing to produce, unlike Microsoft, elegant new hardware to fill the needs of OS X's ever-growing army of users.

Ah well, a BSD developer can always dream, can't he?


By Jordan Hubbard

Jordan Hubbard is a FreeBSD developer.

MORE FROM Jordan Hubbard


Related Topics ------------------------------------------

Apple Business Microsoft