Information Complexity and the Downfall of the Adventure Game

Back in 2005, I wrote a tentative article for The Game Chair titled Information Complexity and the Downfall of the Adventure Game, but due to some annoying legal issues, it never got published there.

A little under a year ago, I contacted my favorite gaming magazine, The Escapist, to see if they’d be interested in publishing it, and they were. It was subsequently featured in issue 116 last September, but I just realized that I never mentioned it here.

If you’d like to read it, you can check it out here.

My Stuff, Open-Sourced

After hearing Chris McAvoy drunkenly wax poetic on how programmers should be more open with the stuff they write—McAvoy himself keeps a lot of his code in a public works repository on Google Code—I decided to work more openly too. So a month or two ago I set up a Mercurial repository at hg.toolness.com and have been regularly pushing a lot of my one-off projects to it. Most of the projects on there don’t mention anything about their license, though, and I’m not sure what to do about that. I hate legal stuff.

My First Mozilla Patch

Last week I decided to try my hand at contributing my first patch to the Mozilla source code repository, and I thought it might be useful to chronicle what went into it, as well as what I learned from the process.

This particular contribution was actually motivated in part by a blog post by Mike Beltzner on the need for Mac-specific improvements to Firefox 3, and also by my own work on the Mac port of open-source Enso.

The Problem

The functionality that Firefox—and, I believe, any Mozilla-based software like Thunderbird—currently lacks is the ability to import and export clipboard and drag-and-drop content in HTML format. What this means in plain english is that, for instance, if you copy “this text” from Firefox and paste it into Mail.app, it gets pasted in as merely “this text”, losing the hyperlink and the bold formatting.

Not only was this inconvenient from an end-user standpoint—Safari and other native OS X widgets deal with this situation fine, as does Firefox on Windows, so it’s entirely expected behavior—but it also prohibited some of Enso’s more interesting commands like map and tex from working with Firefox on OS X. So I thought fixing the problem would be a fun way to get my feet wet with the Mozilla source code, as well as a way to get a first-hand understanding of how the code contribution process works in the Mozilla community.

The Journey

It should first be noted that before I could embark on this quest, I first had to get the source for Mozilla and figure out how to build it, which involved obtaining a basic understanding of how its build system actually worked. I chronicled this back in January with A Simple Mozilla Build Script and Tutorial.

Having set this up, I first asked Myk Melez who I should talk to about fixing the bug. Myk sets at a desk opposite from mine, which makes it really easy for me to completely derail his train of thought with questions like “so, who would I talk to if I wanted to fix this HTML clipboard bug on the Mac?” and “what’s the best Mexican restaurant in Mountain View?”.

Sorry about that, Myk.

In response to that first question, though, Myk told me to talk to Josh Aas, an OS X Mozilla engineer who lives in Philadelphia. I emailed him, and he promptly (and I mean promptly) replied to me and told me exactly what files to look at in Mozilla’s behemoth of a source code tree. Which was extremely helpful.

After browsing the source files for a bit, I had a decent idea of how to proceed, but the first thing I wanted to do was figure out how to display and add logging messages, so that I could easily see what code was being executed. This turned out to be an entire sub-quest in and of itself.

The Logging Sub-quest

My logging sub-quest started by asking Dan Mills—who sits next to Myk and is another hapless victim of my inane questions—how to do it. Dan wasn’t entirely sure, but he told me to check out this page on HTTP debugging for some leads. The clues I needed were there; enabling logging in the Netscape Portable Runtime appeared to involve setting two environment variables like so:

  bash$ export NSPR_LOG_MODULES=nsHttp:5,nsSocketTransport:5,nsHostResolver:5
  bash$ export NSPR_LOG_FILE=/tmp/log.txt

Except I had to figure out what the name of the module was that I wanted to log to. After trying out a few educated guesses—the name of the directory my source files were in, its parent directory, et cetera—I basically decided to do an Mozilla Cross-Reference search on sCocoaLog, which was the name of a variable used by one pre-existing logging statement in the source file I wanted to edit. This ultimately led me to a line of source code that mentioned the code sCocoaLog = PR_NewLogModule(”nsCocoaWidgets”); there was my answer.

Once I had seen my first logging message actually get written to my logfile—and I have to admit that I actually got a really small surge of adrenaline from seeing this—I then had to embark on another sub-quest, which is one I’d never considered before.

The Makefile Sub-quest

As mentioned in my blog post on Mozilla’s build system, Mozilla uses a system of makefiles to build itself. What this means is that if you change or add a single line in Mozilla’s several million lines of code, you just type make and the build system automatically figures out the smallest amount of work it has to do to make a new program that incorporates your change. Now, with any previous project I’ve worked on, whose codebase was well under a million lines of code, the process of “figuring out the smallest amount of work to be done” is fairly quick—only several seconds at most. With Mozilla, though, it took several minutes, which meant that my compile-test-debug-recompile cycle was going to be really long, despite the fact that the actual amount of work (compilation and linking) being done in the compile phase was quite low. I had to short-circuit this process of figuring out what to recompile and just tell the Mozilla build system what to actually compile.

Ordinarily, as the Mozilla Build FAQ says, this would be done by simply giving make the name of the directory containing the source file I had changed. Through trial and error I learned that the code I was changing actually required a two-stage build; by investigating the full output of a standard make, I determined that the command I actually had to use to rebuild Mozilla was this:

  make --directory=widget/src && make --directory=toolkit/library

Thankfully, this build command only took a few seconds to execute.

The Endgame

Actually adding HTML clipboard and drag-and-drop functionality to Mozilla was pretty easy, thanks in part to what I’d learned about Cocoa programming from porting Enso to OS X in December, as well as the invaluable Mozilla Cross-Reference and a blog post on nsITransferable. Once I submitted bug 428096 and my patch for it, Josh did some very vigilant review, noticed some very bad mistakes on my part, and eventually approved the patch once I fixed them.

What I Learned

Let me give you some background.

I’m an only child, and until fairly late in my childhood, I was always one of the only people I knew who was really interested in computer programming. I never had a mentor in this field, so virtually everything I learned was from books and experimentation; it’s unsurprising, then, that I grew up having a strong affinity towards software and development tools that were really well-documented and easy to use on one’s own. This is a big part of why python is my favorite language.

Beyond that, though, there’s also the hacker mantra of “RTFM”, or Read the Fucking Manual, which, while not feedback that I’ve ever given anyone, is nonetheless something I’ve always asked myself before seeking help on a topic. Self-reliance, in other words, is so entrenched in the programming community at large that it’s considered something of a faux pas to ask a question that could be gleaned from experimentation, doing some web research, or reading a book. Since I can discover the answer to pretty much anything on my own through these means—provided I have enough time at my disposal—I rarely ask coding questions in public areas.

What has struck me most about Mozilla since I’ve joined is its unrelenting focus on the concept of community. The thing I really learned while writing my patch was that, through their actions, the people I’ve met in the Mozilla community actually actively encourage inquisitiveness. They constantly ask each other questions; if they know the answer to something, they say so, or they say where the answer can be found, or they say who else in the community might know the answer, or they admit that they have no idea whatsoever. They never say “RTFM”.

Perhaps this openness is simply an extension of the community’s general spirit, or perhaps the community members understand that everyone is a “noob” in some sense because no one single person could possibly be intimately familiar with every component of this enormous project. Whatever the rationale, while I’m not particularly fond of the amount of pre-requisites and knowledge it takes to get to the point of really “getting things done” with the Mozilla codebase, and while the lack of up-to-date documentation for some areas of the source is certainly less than perfect, I am immensely grateful to the developer community for being as helpful, patient, and polite as they are.

PyOhio

I just found out that my hometown of Columbus, Ohio, is holding a Python conference.

This is really cool. I encourage anyone in the area who is interested in computer programming or knows anyone who is to spread the word about this. Aside from being a really fun and easy language to learn and use—it’s simple enough to be taught to children—it’s also powerful enough to be widely used in the industry (for instance, it’s the flagship language of the recently-released Google App Engine).

For those who are unfamiliar with the language, xkcd describes it pretty well, too.

Python as a Platform

Something that recently occurred to me is that the only operating system that doesn’t come with Python pre-installed on it is Windows.

While Linux and OS X both view Python as essentially a first-class development platform–i.e., as something that shrink-wrap applications can be built on–Windows does not. Instead, it’s generally expected that a Python-based Windows application be “frozen”: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into “dependency hell”, it also results in a relatively large download–distributing a simple “Hello World” program is at least a megabyte in size, and makes extending the program’s functionality more difficult.

During the summer of 2007, I made simple Python-based tool for some friends who played World of Warcraft; for those that played on the Mac, the situation was easy because their computers already had Python on them. For the Windows users, however, I was faced with the dilemma of either “freezing” the program into a fairly large download which would be difficult to test and update, or making my friends download the Python installer from python.org and running my Python script from source. Either solution seemed too cumbersome.

So instead, I opted for a different solution: I’d create a tiny executable that would first check for the existence of a Python installation on the end-user’s system. If Python was already installed, the executable would simply extract a Python script from itself and run it. That script was much like ez_setup.py; along with a few tricks, it essentially ensured that my friends would transparently auto-update to the latest version of the tool whenever they launched the program.

If there was no Python installation on the end-user’s system, however, the executable would simply transparently download the latest Python installer from python.org, silently install it on the end-user’s system without any prompting, and then proceed as outlined above.

So, in other words, a small executable file–only about 50k in size–was essentially responsible for “bootstrapping” my program by downloading and silently installing Python if necessary, and then downloading and running the latest version of my program’s source code. This ultimately meant that the user would be able to run a self-updating Python program from a tiny executable without having to click through any installation prompts or dialog boxes.

I’m glossing over a few details here and there are definitely some holes in my implementation–for instance, if the user uses a web proxy, they’re hosed–but I thought the general idea was interesting.

The source for the executable was compiled using the Nullsoft Scriptable Install System, along with its InetLoad plug-in. The source file is below for those interested.

Continue reading ‘Python as a Platform’

On The New York Times

I’ve decided that I’m not a big fan of New York Times, or of newspapers in general. It’s not that they’re fundamentally wrong or anything, but as a source of daily information, it’s complete and total overload for me.

For instance, on the day after Super Tuesday, I wanted to read a bit about what happened, so I tried subscribing to a two-week trial of the online edition of the New York Times on my Kindle. I quickly found myself inundated by a torrent of information that, while perhaps useful to someone else, was useless to me; reading a sentence about how exhausted Obama and Clinton seemed during their speeches once the day was over, or a completely blasé quote from Hillary about how great it was that the American people came out to vote that day, was a complete waste of my attention. The relatively small size of the page on my Kindle, combined with the relatively large text size I had set it to, meant that there were entire pages in the article that had no useful information for me.

Like I said, though, this doesn’t mean they couldn’t be useful to someone else. Indeed, I’ve often found some NY Times technology articles that people have sent me, or articles that involve something I have a deeply vested interest in, to be genuinely informative. But as a whole, every issue of the NY Times is a vast morass of information that’s largely useless to me, and most of my time spent with it is used up wading through the irrelevance to find nuggets of truly useful information.

Fortunately, I’ve recently found an excellent alternative to the kind of information I want to get out of the New York Times: The Economist. It’s published only once a week, and isn’t nearly as large as a daily issue of the Times, let alone a Sunday issue. Its articles are relatively short, but very concise: the information density here is very high, meaning that almost every sentence provides me with something useful to think about. Using the reader’s attention like the precious resource it is, this magazine does an excellent job of informing someone about what’s happening and, more importantly, why they might want to care about it.

A Simple Mozilla Build Script and Tutorial

Last Wednesday, I started working at Mozilla, the awesome company responsible for leading and coordinating the development of the Firefox browser. I’m now working for their labs team, exploring new ways of making the internet easier to use. Everyone I’ve met here so far is very friendly, intelligent, and motivated to make the web a better place; I’m really looking forward to working with them more.

One of the first tasks I’ve given myself here has been to get myself acquainted with the Mozilla build system, which is used to build Firefox, among other Mozilla projects. I’m most familiar with SCons, a Python-based tool we use at Humanized to develop Enso, and while I have some experience with Makefiles and the GNU Autotools, it’s been enlightening to see how the build system for a project as large as Firefox works.

While doing this, I spoke with Mike Meltzner, who passed me some gems of wisdom that were given to him by Vlad Vukicevic regarding a different way of setting up the build system than the one prescribed in the Mozilla Build Documentation.

I thought that the way Mike and Vlad did things was much more preferable than the one prescribed in the traditional build documentation, largely because it kept the CVS checkout 100% “pristine” and used a completely separate, parallel directory structure for everything else; aside from providing a really clean separation between what was under version control and what wasn’t, this gave me a much better idea of how the build system actually worked. Mike said that his way of doing things wasn’t documented anywhere, so I figured I’d write a simple script/tutorial that walks a reader through the setting up of the build system, the checking-out of Firefox from the source code repository, and the building of the application itself. It works on my OS X machine; I’m not sure if it works under cygwin or Linux, but I imagine it should.

Feedback is appreciated.

Continue reading ‘A Simple Mozilla Build Script and Tutorial’

Pickle Presentation Slides

At yesterday’s Chicago Python User Group (ChiPy) meeting, I gave a presentation on the Python pickle module; I’d always been curious in the internals of how serialization works “under the hood” and wanted to learn enough about it to give a talk that was accessible to Python newcomers and veterans alike.

I’d like to thank Robert Zeh of GETCO for hosting the meeting, and everyone for attending–we had a great turnout. Jono DiCarlo gave an awesome presentation on a video game he made with his 7-year old sister, Christopher Allan Webber showcased a really cool animation framework he created called PyStage, and Steve Githens gave a lightning talk on OpenOffice.org’s latest support for Python scripting.

I’ve posted the slides for my talk in PDF format for anyone interested.

PyPy Presentation Slides

At yesterday’s Chicago Python User Group (ChiPy) meeting, I gave an informal presentation on PyPy, a project that I’ve been interested in lately. I’ve posted the slides here as a PDF.

Be warned, though, that PyPy is by no means my area of expertise, so these slides may not be terribly accurate. I’ll be fixing them as I learn about their errors.

Humanized

In late 2005, three colleagues and I founded a company called Humanized. A little over a month ago, we released our first two products, Enso Launcher and Enso Words.

I’m indebted to my incredible co-workers, Jono DiCarlo, Aza Raskin, and Andrew Wilson, for making this possible. Working at Humanized has certainly been the best job I’ve ever had, not to mention an excellent learning experience. Naz Hamid was also instrumental in providing us with superb graphic design and was very helpful as a business mentor in general. Special thanks also go to Andrew Huff, Sten Larsson, Erik Christensen, and Megan Bernard.

If you’d like to learn more about our product line, you may want to take a look at the Enso tour movie.