07.05.07
“The Cathedral and the Bazaar”
There’s a long tradition of great titles in the software engineering world. Djikstra’s “Goto Considered Harmful” has spawned thousands of imitators, and even a meta-paper. Fred Brook’s The Mythical Man-Month clicks as soon as you understand the title. Eric S. Raymond’s “The Cathedral and the Bazaar” gives open source software its defining image.
I read “The Cathedral and the Bazaar” as an introduction to the world of open source software for someone interested in joining it. There’s a history lesson to explain where you’ve come from and what you’ve accomplished. There’s arguments and a case study to show that you’re on the right ship. And finally, the essay serves as a call to arms, to get you excited about becoming a contributor.
I found the history lesson most interesting. I’d had some understanding of this before, but lacked much detail. ESR gives a first-person account of UNIX and software development from the late 1970′s to the present. This is the canonical story of open source. It has its heroes and villains, its true believers and false idols. There’s the Moses figure, Richard Stallman, who freed the users of UNIX from the oppressive yoke of restrictive licenses. But like Moses, he couldn’t enter the promise land. Open source stagnated, awaiting its Last True Prophet. This was Linus Torvalds, who created the Linux kernel, the last piece of the open source operating system.
ESR really uses that of tone. I get the sense that he’s intimidated by Richard Stallman and absolutely idolizes Linus Torvalds. The essay drips with hero worship. Linus is the visionary whose vision he’s writing about.
Beyond the hero worship, there is a clear exposition of the open source model. In order to avoid the problem of N^2 channels of communication amongst N contributors, open-source project have a small set of core developers. These core developers have total control over the project. They decide what gets checked in, and where the project goes. It’s a (hopefully) benevolent oligarchy. Outside of that core, there are occasional contributors and legions of testers, who can submit bug reports. Does this strict hierarchy really sound like a Bazaar?
If you want a real Bazaar, think about Wikipedia. Since I’ve never contributed to an open source project, I kept it in mind as a reference point. It works pretty well, but this perspective has the side effect of making open source development look positively Cathedral-like. Think about it. Rather than having a core set of contributors and legions of users/testers, Wikipedia explicitly aims to make all of its users into contributors. It does this by lowering the barriers to entry as low as it conceivably can, even if this leads to vandalism. All that’s needed to contribute is the ability to write in some language. Last time I checked, English had a few more speakers than C++. Rather than just reporting problems, users are empowered to fix them on the spot. See a typo? Just correct it. Want a citation? Find one and plop it in to help future readers.
I enjoyed “The Cathedral and the Bazaar” for the history lesson, but I find its central image misleading. The development process of open-source projects is as well-organized as any commercial venture.
Craig said,
July 6, 2007 at 4:52 am
Did you read the essay or the book of essays?
Evan M said,
July 6, 2007 at 8:33 am
Linux is an aberration among free software projects (both for its size and its management process). The majority of free software has one or two main developers who either manage all patches contributed by users or at least have the ultimate say-so in whether a commit stays committed.
And there are very few, if any, open source projects that are as well-organized as commercial software. Most are just messes of patches that fly around haphazardly. I think that’s a good thing, though; it’s sort of the only way things can work once you have a sufficient quantity of contributors.
A better (more modern) model is the Perl6 model, where they really will give anyone commit access (I have commit access, for example) and they just use reversion (like wikipedia) to address problems. I think part of the problem historically has been a technical one: that everyone used version control systems that required giving everyone login access to your repository, so you really had to trust people.
Wikipedia’s no panacea. You could make the same argument you made about wikipedia and its admins — anyone is welcome to add to wikipedia but if you disagree with an admin you always lose. A possible free software analogy to the wikipedia model is that anyone is free to submit a patch, but only the committers decide whether it lives. And Wikipedia’s a bad project to try to fork from because they don’t provide the source in any useful format — in the software world it’s like those people who occasionally release a tarball of their code and say “oh yeah, we’re open source, see we gave away some code last november”. That’s sorta like Linksys’s “support” of the Linux kernel (there had to be a lawsuit to get them to release their modifications), or the way Apple used to “support” webkit (the “throw a tarball over the wall” method).
With all that said, Wikipedia’s undoubtedly more open than anyone else, primarily because (I believe) it’s much less harmful to let anyone edit prose than it is code, and because its domain for contributions is much larger. Maybe a more analagous project to Wikipedia would be the entire Gnome system, where there are stewards overseeing which projects are considered part of the desktop (sorta like the RfDs on wikipedia) and anyone can submit their own program for adding if they want. Check out how many projects they have (this doesn’t include all the stuff hosted off-site): http://svn.gnome.org/viewcvs/
ESR is kind of a nutcase so don’t take anything he says too seriously. He’s especially widely mocked for never having written any free software himself. (He brags about fetchmail but it’s pretty crappy software and tends to eat your mail.)
danvk said,
July 6, 2007 at 3:17 pm
Good to hear that ESR is a nut case.
I think you overestimate the importance of admins on Wikipedia. It’s not as though every edit has to be approved by an admin. In fact, almost none are. If you edit a page, only the people who have that page on their Watchlist will notice. And they’re just random, unimportant people like you. There’s very little centralized oversight. You’re right though, in that Wikipedia may be a bit large to compare to a single open source project. Perhaps individual WikiProjects are more analogous.
Are you talking about the Pugs commit bit? I’d heard that Audrey Tang was known for being quite loose with that. Have you contributed anything? Given that Perl 6 has been in development for almost seven years, I’d hesitate to say that it’s a good development model. =)
Leenie said,
July 6, 2007 at 11:25 pm
this is hands-down the best post you’ve ever written, Dan. You should do this for a living. Are you posting this on Amazon?