Java Tool Olympics

Comparing the various Java IDEs

By Will Iverson

The Ground Rules

The objective of this article is to give the readers a sense of the current Java tools. We will look at Sun’s Java Development Kit (JDK) 1.02, Symantec Café DR2, Metrowerks CodeWarrior 9, and Natural Intelligence’s Roaster DR2.1. With the exception of the minimalist JDK 1.02, all of these tools all follow the same basic project system familiar to users of traditional C/C++ environments (from which most of these tools are derived). We will look at these tools, also known as Integrated Development Environments (IDEs), from the perspective of the traditional Edit-Compile-Debug cycle they are based on.

Keep in mind the relative footprints of the various tools, given below. Note that all of the tools made use of up to a few megabytes of system memory during compiles. When looking at the numbers below, keep in mind that they do not include the memory required to launch the various runtimes (typically another 3MB or so, plus a couple of megabytes of System memory).

All tests were performed on an Apple Power Macintosh 7500/100 (100Mhz 601), with 64MB of RAM running System 7.5.3. VM was off.

Editing

The baseline Sun JDK does not include an editor. If you hate yourself, you might want to use SimpleText as your source editor, but if you are on a tight or nonexistent budget you will want to grab BBEdit Lite from your closest InfoMac mirror at <ftp://mirrors.aol.com//pub/info-mac/_Text_Processing/bbe/ bbedit-lite-351.hqx>. All of the other environments feature full source code editors. The following comparative chart identifies features. All of these editors are more than adequate for day to day programming.

fig 1. Roaster Editor

fig 2. Café Editor

fig 3. CodeWarrior Editor

All of the IDE editors support balancing, shifting blocks of text left and right, multifile search and replace, DOS and UNIX linefeeds, and method popup menus. There are subtle differences in the intelligence and flexibility of the various syntax highlighters as demonstrated by fig. 1-3.

All of these editors (with the exception of the JDK) are adequate for most Java development work. On the whole, the differences are relatively minor, but specific features may be of interest to certain developers.

Browsing

Java is an object-oriented language, and as such both graphical and “Smalltalk” style browsers are available in all of the environments save the JDK. While the appearance and ease of use of the various browsers varies widely, the functionality of the browsers is quite similar. All of these browsers require a compile to produce the information necessary to display the browsers.

fig 4. Roaster Browser

If nothing else, the Roaster browser was the most colorful and conveyed the most information in an easy to understand format. It allows pruning of subclasses by clicking on the nexus points, although it did oddly not collapse the layers when the sub-nodes were pruned.

fig 5. CodeWarrior Browser

The CodeWarrior browser also supports pruning, and collapses intelligently when subnodes are removed. Both the CodeWarrior and Café browsers require large amounts of screen space to be even moderately useful. Both the CodeWarrior and Café graphical browsers support type-ahead keyboard navigation.

fig 6. Café Browser

The Café browser offers a high degree of control over configuration-the initially confusing buttons in the upper toolbar allow a user to split and remove panes and define the content of each pane. User pruning is allowed, but the browser does offer an option to only display the sub- and superclasses of the currently selected object.

Compiling

To measure the compilers, we developed three basic tests. We tested the speed of large builds by combining the sources of 39 files in the JDK, and we tested the time to compile and launch a single change to a single file. In addition, we identified five common mistakes and tested the ability of the compilers to generate appropriate error messages.

As a side note, only CodeWarrior and the JDK attempt to support .zip files and standalone applications. Both involve a relatively complex process of copying resources by hand with ResEdit-a laborious and unintuitive process.

Large Builds

The following files were combined into a single project and built: AniArea.java, AppletFrame.java, BounceItem.java, ButtonFilter.java, CardTest.java, ClickArea.java, CLSFractal.java, DelayedSoundArea.java, DitherTest.java, DrawTest.java, GraphicsTest.java, HighlightFilter.java, HrefButtonArea.java, ImageMap.java, ImageMapArea.java, ImageTest.java, JackhammerDuke.java, Matrix3D.java, NameArea.java, NervousText.java, RoundHrefButtonArea.java, SoundArea.java, ThreeD.java, TicTacToe.java, TumbleItem.java, XYZApp.java, Animator.java, ArcTest.java, Blink.java, Chart.java, GraphApplet.java, HighlightArea.java, ImageTape.java, LinkArea.java, MouseTrack.java, RotateFilter.java, RoundButtonFilter.java, SpreadSheet.java.

Small Builds

Using the MoleculeViewer as a reference, the experiences with rapid turnaround varied widely. The goal was to create a project, add the MoleculeViewer files Matrix3D.java and XYZApp.java, build the files, make a minor change (pressing the spacebar once in a comment), and run the applet.

The experience varied wildly. The Metrowerks IDE let me drag the files into the project file, but attempts to run the applet from the IDE failed. The option to build the files as a series of .class files in a folder simply would not allow me to select the Run command (it was permanently grayed out), and the option to build a “Runnable Java Applet” failed to run as well. Dragging and dropping the example1.html file on the Metrowerks Java runtime worked perfectly, but dropping back to the Finder every time I wanted to run my Applet was a serious inconvenience.

Café allowed me to launch directly from the environment, but only after I added the example1.html file to the project - without doing that, it gave me a completely useless error -35. Turning all of the various confirmation dialogs, the time from adding an extra space to a comment at the top of the XYZApp.java file and running through finishing launching the applet took approximately 17 seconds.

Roaster also allowed the launching of applets directly from the environment in roughly 17 seconds as well. What was most interesting about both Café and Roaster was where the slowdown occurred-Café took just under five seconds to compile the file, and spent the rest of the time waiting for the runtime to launch and load the applet. Roaster, conversely, took just under five seconds to launch the runtime, but spent the remainder of the time compiling. In theory, both vendors could chop the turnaround times in at least a third by spending more time working on their shortcomings.

Cafe Test Notes...

Debugger Info turned ON, Optimize Code turned OFF

Note that the SPM is fully threaded (with no way to turn this threading off) both between traditional application processes and internally.

CW9 Test Notes...

Note that the CW9 IDE is threaded (with no way to turn this threading off) between traditional application processes. It would appear from the timing that the same compiler as the JDK is used, with additional time yielded to background processes.

JDK Test Notes...

Debugger Info turned ON, Optimize Code turned OFF, All other checkbox options turned OFF, Disable Threaded Compiles OFF, Partition set to 2430K

Note that with Threaded Compiles turned off, the compiler apparently yielded no time to the system - e.g. the Now QuickDay menubar clock stopped ticking.

Roaster Test Notes...

Debugger Info turned ON

The javac compiler was used. Attempts to build the test suite with the Roaster compiler caused the IDE to quietly terminate. Note that the Roaster IDE is very minimally threaded (with no way to turn this threading off) between traditional application processes. Roaster uses the Javac compiler from Sun. An interesting sidenote - when I compiled files within Roaster with the JDK compiler open, Roaster would send AppleEvents to the JDK and use it to compile.

Error Reporting

All of the IDEs generated errors which are routed to a window. In all of the IDEs, the errors may be doubled-clicked on and the appropriate window will be opened in the editor and the selection set to the appropriate location (the exception to this is the JDK, which opens the file but has no way to communicate the line number. BBEdit Lite allows you to display line number along the left edge of the window, so this was not an insurmountable difficulty).

The five errors we tested were all located in the file XYZApp.java...

1) Commenting out the opening brace on line 49.

2) Commenting out the declaration int ZsortMap[ ]; on line 53.

3) Commenting out the catch(Exception e) { }; block on lines 264-5.

4) Changing if (backBuffer == null) to if (backBuffer = null) on line 334.

5) Removing the arguments of the mat.transform function on line 150.

All of the environments returned appropriate error messages-indeed, the messages were identical, revealing the common code base from which they derive. The only significant difference between the environments was the ability of the CodeWarrior environment to identify the exact location on the line of the error. This is an invaluable feature when deciphering unexpected compiler errors.

Debugging

The baseline Sun JDK does not include a debugger. The implementation of Java from Sun on other platforms is based on TCP/IP, which allows additional capabilities such as remote debugging. All of the other IDEs feature source-level debugging, with a wildly varying set of capabilities.

Metrowerks

The debugger required me to drop the .class file I wished to debug on the debugger application, and then manually drop the .html file on the Metrowerks runtime. I was able to step within a function reasonably well, but stepping in and out had erratic results at best. I was unable to view any useful variables, including the ever-useful this.

Roaster

fig 7. Roaster variable display

The Roaster debugger was by far the most full-featured of the debuggers I reviewed. It displayed variables and the calling chain in floaters (including an interesting popup view of variables-the “floating” view appears when you move the pointer over any field which does not fit in the column). A “Class List” floater allows you to select classes, unload them, and then the interpreter reloads the class when needed-a very interesting step toward a truly dynamic language. The debugger uses the same browser model as the editor, a very convenient capability. Finally, the debugger allows you to edit and recompile the code directly from the debugging pane (with a warning that breakpoints may no longer function).

Café

The Café debugger displayed a stack crawl, and stepping in and out of source code worked well. There was no variable display whatsoever, however, a critical omission. The Café debugger was also the only debugger not to feature a bytecode view.

Project Management

With the exception of the JDK, all of these tools are centered around a project manager of some sort. For many small applets, the project management system is overkill. Disappointingly, all of the IDEs require that you work within this metaphor, despite the fact that Java’s runtime linking completely obviates any need. To simply check the syntax of a source file, the IDEs all require that you first add the file to a project.

For projects involving more than a few files, however, this system works reasonably well. The different project managers look quite similar and perform similar functionality. The chart below highlights the differences between the different environments. As of this writing, the project managers were all satisfactory for projects of up to a few hundred files - more than enough for any Java-based project in the near future.

fig 8. Café Project Manager

The Café Project Manager is by far the most sophisticated of all of the IDEs. It displays Finder icons, allows dragging files both into the environment from the Finder and also drop launching files from inside the project file onto Finder icons. It features built-in Projector source control and nested folders. It allows sorting by any column by clicking on the column name, and multiple option sets.

fig 9. Roaster Package Manager

The Roaster IDE offered far fewer bells and whistles than the Café Project Manager as the rather sparse display in fig. 9 will attest. Roaster is unusually intelligent in one very important way, however. It will automatically group files by package statement, a very intelligent and useful feature. To demonstrate this, I added package myPackage; statements to the two .java files, and Roaster automatically moved the files into the hierarchy displayed above-no user interaction beyond making the project was required.

fig 10. Metrowerks Project Manager

The Metrowerks IDE sparse but straightforward project management. Source control is available with CodeManager as an add-on product, with commands executed via ToolServer.

Java Runtime

Also referred to as the interpreter, the Java Runtime is the portion of the Java package which executes the bytecode generated by the compiler, transforming the bytecode into native code. Each environment includes a different runtime, and as of this writing Apple had shipped the Mac OS Runtime for Java 1.0a1. All of these runtimes toss files all over your System Folder and your Extensions Folder, creating a potential support nightmare. If and when Apple integrates a common implementation of Java into the operating system, the issue of which runtime is used will become moot. As the focus of this review is on the tools, space limitations prevent extensive reviews of the performance of the various runtimes. All of the runtimes require more than 2MB and less than 5MB of space to run,

Default Partition

Café 5MB

Roaster 3MB

JDK 2.5MB

CodeWarrior 3.5MB

Apple (MRJ 1.0a1) 1MB

All of the environments allocate a fair amount of additional temp memory (which appears as System Software under About This Macintosh. Typically, the runtimes would allocate at a minimum 2-3MB of additional memory, and potentially up to 10MB of additional temp mem. Anecdotally, there was a fairly high correlation between the amount of memory available to the runtime and the performance. Note that the Café default partition appeared to be set high-it appeared to behave quite reasonably when set to the suggested minimum of 3MB.

It is necessary to clarify a potentially confusing bit of terminology. Some people confuse a Just-in-time (JIT) compiler with a “native” compiler. The simplest implementation of a runtime converts bytecode serially, “forgetting” code as soon as it executes. A JIT runtime remembers what it has converted and dispatches intelligently directly to the converted code when it encounters a method it has already compiled. In theory, a JIT could perform runtime optimization specific to the processor it is executing on, thereby outperforming traditionally compiled code. The reality, of course, is that the current best case for a simple runtime is likely to be roughly equal to the performance of the original 68K emulator, with the best case for a JIT emulator near that of the DR emulators introduced with the PCI Power Macintosh and SpeedDoubler.

The Winner Is...

Basically, the true victor is the consumer. All of these environments will do the job quite reasonably, with three vendors fighting tooth-and-nail over this market. Of all of the environments, the Roaster environment shows the greatest number of Java-specific innovations. Café is the most polished of the three, barring the debugger. The CodeWarrior Java implementation is the least impressive from a tool perspective-Java is “just another” language, like Pascal or C/C++, but it has the greatest attention to detail for deployment. All three are crippled in various ways, and all have certain golden spots. I encourage you to explore the vendor’s web sites for more information, and if possible to purchase all three to make your own evaluation-or to wait until they mature.

What I’d Like To See

After spending quite a bit of time with all of these environments, what struck me over and over was the lack of significant timesaving features and basic functionality. I’ve organized these gripes and requests into two basic categories, one directed at Apple and the other at the tools vendors.

Apple

• A single, robust, easy-to-use and install high-performance runtime, installed by default by a standard System installation as a single shared library.

• Easy, straightforward calling conventions from Java to C and back, at least as easy to use as the Mixed Mode interfaces.

• Macintosh Toolbox interfaces

• Easy, consistent method of developing standalone applications. I didn’t delve into this aspect of the tools at length, because the process is somewhat tricky and varies depending on vendor. This would most likely take the form of a new resource type. Compilers could be set to pipe bytecode output as resources, which would then automatically be merged into the application by the various project managers. This would allow developers to mix native PowerPC, 68K, and Java code all within a single application file.

Tools Vendors

• The ability to compile Java source without a project file.

• More Java specific support throughout, such as Roaster’s package support.

• C++ to Java conversion tools

• The ability to generate native 68K and PowerPC code instead of bytecode

The thing which strikes me over and over about Java is the vast discrepancy between the potential and the reality. All of the tools have serious rough edges which make them feel as though they were rushed to market (which they were). None of the tools are geared toward serious development-they are all adequate for relatively trivial applet development, but not real Macintosh applications. We are closer to the next evolutionary step from C++ today than at any time since Dylan-let’s not screw it up, OK?

For More Information

Symantec Café http://café.symantec.com/

Natural Intelligence Roaster http://www.natural.com/

Metrowerks CodeWarrior http://www.metrowerks.com/

Sun JDK http://www.javasoft.com/