The Veteran Neophyte: The Right Tool for the Job

Dave Johnson

Dynamic programming languages are cool. Once you've tasted dynamic programming, it's hard to go back to the old, crusty, static way of doing things. But the fact remains that almost all commercial software is still written with static languages. Why?

Recently I took a class in Newton programming. For me personally the Newton isn't a very useful device, only because I never carry around a notepad or calendar or address book or to-do list and I don't have a need to collect any sort of data out in the field. But even though it's not terribly useful to me, it is very useful to a lot of people -- and useful or not, it's a really cool device. Programming the Newton, for those of you who haven't had the pleasure, is very, very different from programming the Macintosh in C or C++ or Pascal, and is incredibly attractive in a lot of ways.

The language that you use to program the Newton, NewtonScript, is an example of an object-oriented dynamic language, or OODL. (See? Even the acronym is cool.) This means a number of things, but the upshot is that it's very programmer-friendly and very flexible. Now, I don't pretend to be an expert in languages, not by a long shot, so I can't offer any incisive comparisons with other "modern" languages, but I can tell you what it feels like for a dyed-in-the-wool C programmer to leap into this new and different world. It feels great.

One well-known feature of dynamic languages is garbage collection, the automatic management of memory. Objects in memory that are no longer needed are automatically freed, and in fact there is no way to explicitly free them other than making sure that there are no references to them any more, so that the garbage collector can do its thing. I didn't fully realize how much time and effort and code it takes to deal with memory management until I didn't have to do it anymore. There's something almost naughty about it, going around cavalierly creating objects in memory without worrying about what to do with them later. After a lifetime of living in mortal fear of memory leaks, it feels deliciously irresponsible. I like it. I like it a lot.

NewtonScript's object model is refreshingly simple and consistent. There are the usual "simple" data types -- integers, real numbers, Booleans, strings, and so on -- and only two kinds of compound objects: arrays and frames. An array, as you might expect, is simply a linear, ordered group of objects, and the individual objects are referenced by their index (their position in the array). Frames are an unordered collection of items in named slots; you refer to a particular item by the name of its slot. Frames are also the only NewtonScript objects that can be sent messages, and the message is simply the name of a slot that contains a function.

Because NewtonScript is dynamic, variables or frame slots or array members can hold any kind of data, including other arrays or frames, or even functions, and the kind of data can be changed at any time. The size of the array or frame can be changed anytime, too; you can add or delete items as needed, without worrying about managing the changing memory requirements. This kind of flexibility is a big chunk of what makes dynamic languages so, well, dynamic. Such a thing is of course unimaginable in a static language, where each byte must be explicitly allocated before it's needed, carefully tracked while used, and explicitly deallocated when you're done with it.

NewtonScript is also introspective, meaning that all objects "know" all about themselves. (Isn't that a nice term? I like the idea of a language being introspective -- sitting there, chin in hand, pondering itself.) The type of a piece of data is stored with the data, and named items keep their names. In fact, everything in memory is coherent, with a well-defined identity; there is no possibility of undifferentiated bits getting schlepped around, no possibility of a dangling pointer or a string being interpreted as a real number. In static languages, of course, all that design-level information is thrown out at compile time, and doesn't exist in the running program at all. There's nothing but undifferentiated bits, really. What a mess.

And that means that debugging, for the most part, has to take place at the machine level. By the time the program is running, it's just a maze of pointers and bytes and instructions, fine for a machine but nasty for humans. Of course, to combat this we have elaborate, complex programs called source-level debuggers. They give you the sense that the names still exist, thank goodness, but it's just a trick, and depends on an external file that correlates symbols with locations in memory. If you don't have the symbol file, you're out of luck. (Confession time: In my regular C programming I avoid low-level debugging like the plague. Usually I'd rather spend an hour in a source-level debugger than spend five minutes in MacsBug -- I know, I know, I'm a wimp -- precisely because all the information that helps me to think about my program, the names and so on, still "exist" in the source-level debugger. In NewtonScript, there isn't even such a thing as low-level debugging! All that design information is right there in the guts of the running program. Hallelujah!)

With dynamic languages like NewtonScript, you can let go of the details of the machine's operation, and deal with your program's operation instead -- you can think at the design level, not the machine level. And it's an incredible relief to float free of the bits and bytes and pointers and handles and memory leaks and messy bookkeeping. Most of the ponderous baggage that comes along with writing a computer program goes away. I mean really, how much longer must we approach the machine on its terms when we want to build something on it? Users were released from that kind of bondage to the machine's way of doing things long ago. So what are we waiting for? Obviously we can't program the Macintosh in NewtonScript (more's the pity) but why aren't we all chucking our C++ compilers in exchange for Prograph or Lisp or Smalltalk or Dylan? Well, some of us are. But I think there are two major hurdles to overcome before dynamic languages become mainstream: the need for speed, and inertia.

Dynamic languages carry their own baggage, of course. In the same way that making the Macintosh easier for people to use made it harder to program because the complexity and bookkeeping were shunted behind the scenes, making programming languages easier to use also requires new behind-the-scenes infrastructure and complexity. (Somebody has to do the memory management, after all.) This usually results in a bigger memory footprint and slower execution. For "normal" operations, we're long past the point where that mattered: the hardware is beefy enough to handle it without blinking. But software always pushes the limits of the hardware. Consequently, there are still times when it's important to squeeze every drop of performance out of the machine. And dynamic languages are just not very good at that. (I don't think you'd want to write your QuickDraw 3D renderer in Lisp.) So any dynamic language that hopes for mainstream commercial acceptance had better have a facility for running hunks of "external" code. That way you could write the bulk of your program in a dynamic language, but still be able to write any time-critical parts in your favorite static language and plug them in. You'd lose the protection of the dynamic language when running the external code, but that's a reasonable tradeoff.

Inertia is the other big problem. People, once they know one way to do something, are often loath to change it, especially if they've been doing it that way for a long time. I'm guilty of this in my own small way: every time I learn a spiffy, liberating new way to program I think I'll never go back to the "old" way. But the next time I set off to write some code I automatically reach for the familiar tools, not the new ones. (Lucky for me, the only way to program the Newton is in NewtonScript.) Fortunately, neither one of these hurdles will stop the evolution of our tools. It's unstoppable, if perhaps slower than we might like. There's already a whole spectrum of tools available. From Assembler to AppleScript, Pascal to Prograph, there are tools that allow anyone with enough interest to teach their computers to do new things. The line between users and programmers continues to blur, and dynamic languages can only help that process. I love the thought of putting programming tools into the hands of "nonprogrammers" -- kids, artists, hobbyists -- and seeing what they come up with. You can bet it will be something new, something that people tied to the machine would never have thought of. I can't wait.

---

RECOMMENDED READING

• Unleashed: Poems by Writers' Dogs, edited by Amy Hempel and Jim Shepard (Crown, 1995).

DAVE JOHNSON recently enrolled his smallest dog -- named Io (eye-oh) but affectionately called The Stinklet -- in an agility class. Dog agility is a sort of obstacle course for dogs, with ramps and jumps and tunnels and poles to climb and leap over and crawl and weave through. Dave got so involved that he started building agility courses in the living room. He came to his senses, thankfully, before creating any permanent installations.

Dave is easing up on his working life: beginning with the next issue, he'll be working 3/4 time. He had to give up some things, and it was decided (reasonably enough) that helping to edit the rest of develop was more important than writing this column. Look for guest Neophytes in coming issues, with perhaps the occasional installment from Dave.

Thanks to Lorraine Anderson, Jeff Barbose, Paul Dreyfus, Bo3b Johnson, Lisa Jongewaard, and Ned van Alstyne for their always enlightening review comments.

Dave welcomes feedback on his musings. He can be reached at JOHNSON.DK on AppleLink or eWorld, or dkj@apple.com on the Internet.